Gaussian State Space Models Research Articles

Bayesian-Learning-Based Diffusion Least Mean Square Algorithms Over Networks.

To improve the learning performance of the conventional diffusion least mean square (DLMS) algorithms, this article proposes Bayesian-learning-based DLMS (BL-DLMS) algorithms. First, the proposed BL-DLMS algorithms are inferred from a Gaussian state-space model-based Bayesian learning perspective. By performing Bayesian inference in the given Gaussian state-space model, a variable step-size and an estimation of the uncertainty of information of interest at each node are obtained for the proposed BL-DLMS algorithms. Next, a control method at each node is designed to improve the tracking performance of the proposed BL-DLMS algorithms in the sudden change scenario. Then, a lower bound on the variable step-size of each node of the proposed BL-DLMS algorithms is derived to maintain the optimal steady-state performance in the nonstationary scenario (unknown parameter vector of interest is time-varying). Afterward, the mean stability and the transient and steady-state mean square performance of the proposed BL-DLMS algorithms are analyzed in the nonstationary scenario. In addition, two Bayesian-learning-based diffusion bias-compensated LMS algorithms are proposed to handle the noisy inputs. Finally, the superior learning performance of the proposed learning algorithms is verified by numerical simulations, and the simulated results are in good agreement with the theoretical results.

Individualized survival predictions using state space model with longitudinal and survival data.

Monitoring disease progression often involves tracking biomarker measurements over time. Joint models (JMs) for longitudinal and survival data provide a framework to explore the relationship between time-varying biomarkers and patients' event outcomes, offering the potential for personalized survival predictions. In this article, we introduce the linear state space dynamic survival model for handling longitudinal and survival data. This model enhances the traditional linear Gaussian state space model by including survival data. It differs from the conventional JMs by offering an alternative interpretation via differential or difference equations, eliminating the need for creating a design matrix. To showcase the model's effectiveness, we conduct a simulation case study, emphasizing its performance under conditions of limited observed measurements. We also apply the proposed model to a dataset of pulmonary arterial hypertension patients, demonstrating its potential for enhanced survival predictions when compared with conventional risk scores.

Developing a dynamic quality prediction model for limited samples target grade based on transfer learning

In response to the dynamic demands of the market, this study addresses the challenge of frequent operational adjustments in a production line to accommodate diverse product grades. The resulting scarcity of data in the new operating conditions (target grade) impedes the development of reliable soft-sensor prediction models. A two-step learning method, termed S2-LGMNSSM-TS-T, is proposed. This method employs a semi-supervised latent Gaussian mixture nonlinear state space model (S2-LGMNSSM-TS) trained on both target and source data, providing a dynamic, one-step-ahead predictive soft sensor. To overcome data scarcity in the target grade, insights from the source are utilized. With a Gaussian mixture prior distribution, S2-LGMNSSM-TS identifies dynamic behaviors in both target and source grades. The enhanced S2-LGMNSSM-TS-T configuration focuses on target-grade predictions, leveraging source knowledge and mitigating data scarcity issues. A numerical example and an industrial case study demonstrate the model's effectiveness in improving target-grade predictions through source knowledge utilization.

Application of A Linear Gaussian State Space Model and The Kalman Filter in Estimating Expectations of The Natural Logarithmic Monthly Returns of Shiba Cryptocurrency

Application of A Linear Gaussian State Space Model and The Kalman Filter in Estimating Expectations of The Natural Logarithmic Monthly Returns of Shiba Cryptocurrency

A novel robust multivariate Laplace distribution-based distributed consensus information fusion

The multivariate Laplace distribution (MLD) possesses a lesser set of prior parameters compared to the Student’s t-distribution, thus simplifying the modeling of measurement data subject to heavy-tailed noise corruption. This short communication introduces a novel distributed hybrid consensus fusion algorithm utilizing the MLD modeling and variational Bayesian approach. Firstly, we formulate the measurement noise associated with individual sensor nodes and introduce the hierarchical Gaussian form of MLD to enhance the feasibility of variational inference. Secondly, we opt for the inverse-Wishart (IW) distribution to serve as the conjugate prior to the positive definite matrix, enabling adaptive parameter learning within the MLD context. As a result, a novel hierarchical Gaussian state–space model (SSM) is established for multi-sensors network. Subsequently, we derive a novel local information filter to acquire higher precision information and innovation vector–matrix pairs, outperforming the Student’s t distribution-based filter. These derivations are grounded in the new hierarchical SSM, and the vector–matrix pairs aid in information fusion across the network. Lastly, we present a distributed robust consensus filter for fusing local information using a hybrid consensus approach that devoid of a central fusion node. Simulation results demonstrate that the proposed consensus filter can obtain better estimation accuracy under the interference of heavy-tailed measurement outliers.

AI‐guided patient stratification for neurodegenerative disorders using unsupervised trajectory modelling

AbstractBackgroundPredicting neurodegenerative disorders early has major implications for timely clinical management and patient outcomes. Despite advances in medical technology, we still lack tools to precisely stratify patients for targeted interventions. Here, we propose an unsupervised modelling approach based on mixtures of state space models that learns from longitudinal multimodal (brain imaging, cognitive) data to stratify patients into clusters with different clinical outcomes. Our approach has the potential to support dementia prediction based on cost‐effective, non‐invasive digital measures alone.MethodWe trained (with cross‐validation, 4 components selected with elbow method) a mixture of linear Gaussian state space models on 571 trajectories (2‐4 assessments) from ADNI comprising a) state variables: neuroimaging biomarkers (temporal lobe gray matter density, β‐amyloid) and b) observations: cognitive tests (ADNI‐Mem, ADNI‐EF, MoCA, & ADAS‐13) collected at regular 2‐year intervals. The model clustered individuals based on longitudinal changes in biomarkers and the relationship between biomarkers and cognitive scores (figure 1). We validated model‐derived clusters against clinical outcomes (unseen by the model) based on each individual’s final assessment.ResultOur unsupervised modelling approach effectively stratifies individuals by clinical outcome. Cluster A comprises 65% healthy controls and 33% stable MCI patients (individuals with MCI who do not convert to AD in a 3‐year period) while cluster D comprises 77% Alzheimer’s patients (table 1). Profiling these clusters using MMSE score validates our approach: individuals in cluster A maintain high memory scores over 6 years whereas individuals in cluster D decline (figure 2). Further, we find that our model maintains good stratification when tested a) with a single rather than multiple assessments and b) with cognitive data only (i.e. without biomarkers). These results enhance the clinical utility of our approach, as collecting longitudinal and biomarker data may prove challenging in clinical settings.ConclusionOur modelling approach robustly stratifies patients by clinical outcome when trained on longitudinal multimodal data and tested using cognitive scores alone. Our approach has strong potential to generalize to other non‐invasive and cost‐effective data (e.g. digital markers from wearable technologies), enhancing the translational impact of our AI‐guided tool for early and precise patient stratification in clinical practice.

Component-wise iterative ensemble Kalman inversion for static Bayesian models with unknown measurement error covariance

The ensemble Kalman filter (EnKF) is a Monte Carlo approximation of the Kalman filter for high dimensional linear Gaussian state space models. EnKF methods have also been developed for parameter inference of static Bayesian models with a Gaussian likelihood, in a way that is analogous to likelihood tempering sequential Monte Carlo (SMC). These methods are commonly referred to as ensemble Kalman inversion (EKI). Unlike SMC, the inference from EKI is asymptotically biased if the likelihood is non-linear and/or non-Gaussian and if the priors are non-Gaussian. However, it is significantly faster to run. Currently, a large limitation of EKI methods is that the covariance of the measurement error is assumed to be fully known. We develop a new method, which we call component-wise iterative EKI (CW-IEKI), that allows elements of the covariance matrix to be inferred alongside the model parameters at negligible extra cost. This novel method is compared to SMC on a linear Gaussian example as well as four examples with non-linear dynamics (i.e. non-linear function of the model parameters). The non-linear examples include a set of population models applied to synthetic data, a model of nitrogen mineralisation in soil that is based on the Agricultural Production Systems Simulator, a model predicting seagrass decline due to stress from water temperature and light, and a model predicting coral calcification rates. On our examples, we find that CW-IEKI has relatively similar predictive performance to SMC, albeit with greater uncertainty, and it has a significantly faster run time.

A Comparison Study of Predictive Models for Electricity Demand in a Diverse Urban Environment

The increasing population migration to urban and peri-urban areas increases basic service needs for cities worldwide. Residential electricity demand increases with more customers and varies with novel uses, such as charging electric vehicles, which may add additional dynamics to the residential electricity demand profile. Widespread installation of smart electricity meters provides fine temporal resolution data reflecting current demands and supports predicting future demands. As part of a demand-side management program, understanding the main drivers of current electricity usage based on demand-driven and exogenous predictors represents a valuable tool for utilities facing new demand scenarios. This work presents the application of multiple models to forecast electricity demand based on the input data and the forecasting horizon. Models with exogenous variables as predictors are part of the input–output category, including a Feed Forward Neural Network, Random Forest, and a Linear Gaussian State Space model. The second category is demand-driven models, where predictors include only previous demand values. The demand-driven models in our analysis include a univariate Nonlinear Autoregressive Neural Network and a Linear Gaussian State Space. Using smart electricity meter data from the greater Chicago area, we compare the performance of the models on two different types of accounts: single- and multi-family residential users when forecasting one and multiple steps. Results show that the Linear Gaussian model reports an R2 of 0.99 compared to an average R2 of 0.92 from the Feed Forward Neural Networks and Random Forest when forecasting single- and multi-family electricity demand one step ahead. However, Nonlinear Autoregressive Neural Networks report an average R2 of 0.85 compared to the Linear Gaussian R2 of 0.58 when forecasting 48 steps. We also found that the most important predictors for single-family demand are temporal variables like weekdays, working and non-working days, and day hours. For multi-family demand, electricity demand at the same hour as the previous week replaces weekdays as a significant predictor. Different forecasting models can assist utilities and city planners to manage demand under different and novel residential electricity usage conditions.

Student’s t-Based Robust Poisson Multi-Bernoulli Mixture Filter under Heavy-Tailed Process and Measurement Noises

A novel Student’s t-based robust Poisson multi-Bernoulli mixture (PMBM) filter is proposed to effectively perform multi-target tracking under heavy-tailed process and measurement noises. To cope with the common scenario where the process and measurement noises possess different heavy-tailed degrees, the proposed filter models this noise as two Student’s t-distributions with different degrees of freedom. Furthermore, this method considers that the scale matrix of the one-step predictive probability density function is unknown and models it as an inverse-Wishart distribution to mitigate the influence of heavy-tailed process noise. A closed-form recursion of the PMBM filter for propagating the approximated Gaussian-based PMBM posterior density is derived by introducing the variational Bayesian approach and a hierarchical Gaussian state-space model. The overall performance improvement is demonstrated through three simulations.

Precision-based sampling for state space models that have no measurement error

This article presents a computationally efficient approach to sample from Gaussian state space models. The method is an instance of precision-based sampling methods that operate on the inverse variance-covariance matrix of the states (also known as precision). The novelty is to handle cases where the observables are modeled as a linear combination of the states without measurement error. In this case, the posterior variance of the states is singular and precision is ill-defined. As in other instances of precision-based sampling, computational gains are considerable. Relevant applications include trend-cycle decompositions, (mixed-frequency) VARs with missing variables and DSGE models.

Bayesian multivariate nonlinear state space copula models

A novel flexible class of multivariate nonlinear non-Gaussian state space models, based on copulas, is proposed. Specifically, it is assumed that the observation equation and the state equation are defined by copula families that are not necessarily equal. Inference is performed within the Bayesian framework, using the Hamiltonian Monte Carlo method. Simulation studies show that the proposed copula-based approach is extremely flexible, since it is able to describe a wide range of dependence structures and, at the same time, allows us to deal with missing data. The application to atmospheric pollutant measurement data shows that the approach is suitable for accurate modeling and prediction of data dynamics in the presence of missing values. Comparison to a Gaussian linear state space model and to Bayesian additive regression trees shows the superior performance of the proposed model with respect to predictive accuracy.

Fine-grained Network Traffic Prediction from Coarse Data

ICT systems provide detailed information on computer network traffic. However, due to storage limitations, some of the information on past traffic is often only retained in an aggregated form. In this paper we show that Linear Gaussian State Space Models yield simple yet effective methods to make predictions based on time series at different aggregation levels. The models link coarse-grained and fine-grained time series to a single model that is able to provide fine-grained predictions. Our numerical experiments show up to 3.7 times improvement in expected mean absolute forecast error when forecasts are made using, instead of ignoring, additional coarse-grained observations. The forecasts are obtained in a Bayesian formulation of the model, which allows for provisioning of a traffic prediction service with highly informative priors obtained from coarse-grained historical data.

Robust IMM Filtering Approach with Adaptive Estimation of Measurement Loss Probability for Surface Target Tracking

A suitable jump Markov system (JMS) filtering approach provides an efficient technique for tracking surface targets. In complex surface target tracking situations, due to the joint influences of lost measurements with an unknown probability and heavy-tailed measurement noise (HTMN), the estimation accuracy of conventional interacting multiple model (IMM) methods may be seriously degraded. Aiming to address the filtering issues in JMSs with HTMNs and random measurement losses, this paper presents an IMM filtering approach with the adaptive estimation of unknown measurement loss probability. In this study, we assumed that the measurement noises obey student’s t-distributions and then proposed Bernoulli random variables (BRVs) to characterize the random measurement loss. Notably, by converting the two likelihood functions from the weighted sum form to exponential multiplication, we established hierarchical Gaussian state space models to directly utilize the variational inference method. The system state vectors, unknown distribution parameters, BRVs, and unknown measurement loss probabilities were estimated simultaneously according to the variational Bayesian inference in the IMM framework. The results of maneuvering target tracking simulations verified that the presented filtering approach demonstrated superior estimation accuracy compared to existing IMM filters.

High-dimensional conditionally Gaussian state space models with missing data

We develop an efficient sampling approach for handling complex missing data patterns and a large number of missing observations in conditionally Gaussian state space models. Two important examples are dynamic factor models with unbalanced datasets and large Bayesian VARs with variables in multiple frequencies. A key observation underlying the proposed approach is that the joint distribution of the missing data conditional on the observed data is Gaussian. Furthermore, the inverse covariance or precision matrix of this conditional distribution is sparse, and this special structure can be exploited to substantially speed up computations. We illustrate the methodology using two empirical applications. The first application combines quarterly, monthly and weekly data using a large Bayesian VAR to produce weekly GDP estimates. In the second application, we extract latent factors from unbalanced datasets involving over a hundred monthly variables via a dynamic factor model with stochastic volatility.

Process monitoring using recurrent Kalman variational auto-encoder for general complex dynamic processes

Recently, latent models have continued to find advantages in statistical process monitoring, especially with multifarious extensions coping with the nonlinearity and dynamics confronted by the pattern extraction. However, when later performing the famous T2 statistic on the patterns, the issue of non-Gaussianity has rarely been focused alongside, hence vitiating the control of Type-I and -II error in practice. In this paper, a model, in the name of recurrent Kalman variational auto-encoder (RKVAE), is theoretically derived for handling these properties simultaneously, and proposed for its use on monitoring general complex processes. The main idea is to build a monitoring panel on the basis of further reasoning the dynamics underlying the nonlinearly extracted Gaussian features. The model hierarchically consists of three parts: the vanilla VAE for encoding latent Gaussian features, a linear Gaussian state-space model (LGSSM) for temporally reasoning these features, and an RNN for recurrently updating the online LGSSM by fusing several candidate parameters. For the latter K in the name, it is the Kalman filter (KF) that facilitates the feature noise filtering and LGSSM identification. The model has, structurally, a good capacity to be extensible from both the depth and width when modelling complex processes. Finally, based on the explicit formula of LGSSM, a detection panel is designed, and two case studies are given to demonstrate its effectiveness in monitoring and relevant analysis.

A Novel Robust IMM Filtering Method for Surface-Maneuvering Target Tracking with Random Measurement Delay

A proper filtering method for jump Markov system (JMS) is an effective approach for tracking a maneuvering target. Since the coexisting of heavy-tailed measurement noises (HTMNs) and one-step random measurement delay (OSRMD) in the complex scenarios of the surface maneuvering target tracking, the effectiveness of typical interacting multiple model (IMM) techniques may decline severely. To solve the state estimation problem in JMSs with HTMN and OSRMD simultaneously, this article designs a novel robust IMM filter utilizing the variational Bayesian (VB) inference framework. This algorithm models the HTMNs as student’s t-distribuitons, and presents a random Bernoulli variable to describe the OSRMD in JMSs. By transforming measurement likelihood function form from weighted summation to exponential product, this paper constructs hierarchical Gaussian state space models. Then, the state vectors, random Bernoulli vairable, and model probability are inferred jointly according to VB inference. The surface maneuvering target tracking simulation example result indicates that the presented IMM filter achieves superior target state estimation accuracy among existing IMM filters.

Tracking Brand-Associated Polarity-Bearing Topics in User Reviews

Abstract Monitoring online customer reviews is important for business organizations to measure customer satisfaction and better manage their reputations. In this paper, we propose a novel dynamic Brand-Topic Model (dBTM) which is able to automatically detect and track brand-associated sentiment scores and polarity-bearing topics from product reviews organized in temporally ordered time intervals. dBTM models the evolution of the latent brand polarity scores and the topic-word distributions over time by Gaussian state space models. It also incorporates a meta learning strategy to control the update of the topic-word distribution in each time interval in order to ensure smooth topic transitions and better brand score predictions. It has been evaluated on a dataset constructed from MakeupAlley reviews and a hotel review dataset. Experimental results show that dBTM outperforms a number of competitive baselines in brand ranking, achieving a good balance of topic coherence and uniqueness, and extracting well-separated polarity-bearing topics across time intervals.1

Numerical Gaussian Process Kalman Filtering for Spatiotemporal Systems

We present a novel Kalman filter for spatiotemporal systems called the numerical Gaussian process Kalman filter (NGPKF). Numerical Gaussian processes have recently been introduced as a physics informed machine learning method for simulating time-dependent partial differential equations without the need for spatial discretization while also providing uncertainty quantification of the simulation resulting from noisy initial data. We formulate numerical Gaussian processes as linear Gaussian state space models. This allows us to derive the recursive Kalman filter algorithm under the numerical Gaussian process state space model. Using two case studies, we show that the NGPKF is more accurate and robust, given enough measurements, than a spatial discretization based KF.

Time series clustering with an EM algorithm for mixtures of linear Gaussian state space models

In this paper, we consider the task of clustering a set of individual time series while modeling each cluster, that is, model-based time series clustering. The task requires a parametric model with sufficient flexibility to describe the dynamics in various time series. To address this problem, we propose a novel model-based time series clustering method with mixtures of linear Gaussian state space models, which have high flexibility. The proposed method uses a new expectation-maximization algorithm for the mixture model to estimate the model parameters, and determines the number of clusters using the Bayesian information criterion. Experiments on a simulated dataset demonstrate the effectiveness of the method in clustering, parameter estimation, and model selection. The method is applied to real datasets commonly used to evaluate time series clustering methods. Results showed that the proposed method produces clustering results that are as accurate or more accurate than those obtained using previous methods.

An intrinsically motivated learning algorithm based on Bayesian surprise for cognitive radar in autonomous vehicles

IntroductionThis paper proposes a Bayesian surprise learning algorithm that internally motivates the cognitive radar to estimate a target's state (i.e., velocity, distance) from noisy measurements and make decisions to reduce the estimation error gradually. The work exhibits how the sensor learns from experiences, anticipates future responses, and adjusts its waveform parameters to achieve informative measurements based on the Bayesian surprise.MethodsFor a simple vehicle-following scenario where the radar measurements are generated from linear Gaussian state-space models, the article adopts the Kalman filter to carry out state estimation. According to the information within the filter's estimate, the sensor intrinsically assigns a surprise-based reward value to the immediate past action and updates the value-to-go function. Through a series of hypothetical steps, the cognitive radar considers the impact of future transmissions for a prescribed set of waveforms–available from the sensor profile library–to improve the estimation process.Results and discussionNumerous experiments investigate the performance of the proposed design for various surprise-based reward expressions. The robustness of the proposed method is compared to the state-of-the-art for practical and risky driving situations. Results show that the reward functions inspired by estimation credibility measures outperform their competitors when one-step planning is considered. Simulation results also indicate that multiple-step planning does not necessarily lead to lower error, particularly when the environment changes abruptly.