Bayesian System Identification Research Articles

Probabilistic neural transfer function estimation with Bayesian system identification.

Neural population responses in sensory systems are driven by external physical stimuli. This stimulus-response relationship is typically characterized by receptive fields, which have been estimated by neural system identification approaches. Such models usually require a large amount of training data, yet, the recording time for animal experiments is limited, giving rise to epistemic uncertainty for the learned neural transfer functions. While deep neural network models have demonstrated excellent power on neural prediction, they usually do not provide the uncertainty of the resulting neural representations and derived statistics, such as most exciting inputs (MEIs), from in silico experiments. Here, we present a Bayesian system identification approach to predict neural responses to visual stimuli, and explore whether explicitly modeling network weight variability can be beneficial for identifying neural response properties. To this end, we use variational inference to estimate the posterior distribution of each model weight given the training data. Tests with different neural datasets demonstrate that this method can achieve higher or comparable performance on neural prediction, with a much higher data efficiency compared to Monte Carlo dropout methods and traditional models using point estimates of the model parameters. At the same time, our variational method provides us with an effectively infinite ensemble, avoiding the idiosyncrasy of any single model, to generate MEIs. It allows to estimate the uncertainty of stimulus-response function, which we have found to be negatively correlated with the predictive performance at model level and may serve to evaluate models. Furthermore, our approach enables us to identify response properties with credible intervals and to determine whether the inferred features are meaningful by performing statistical tests on MEIs. Finally, in silico experiments show that our model generates stimuli driving neuronal activity significantly better than traditional models in the limited-data regime.

Scenario based optimisation over uncertain system identification models

Models of dynamical systems are often built from observed data using system identification techniques, and the uncertainty in the model parameters is commonly described using confidence regions to which the true model parameters belong with a certain probability. In this paper we propose methods which use the scenario approach to solve robust optimisation problems for model based design where there is uncertainty in the model parameters. The methods draw samples (scenarios) of the uncertain model parameters and solve an optimisation problem only involving the drawn scenarios. Both a Bayesian system identification setting where the model parameters are stochastic, and a non-Bayesian system identification setting where the model parameters are treated as being deterministic are considered. The Bayesian and non-Bayesian settings require different treatment, and whereas standard scenario optimisation theory can be directly utilised in the Bayesian setting, this is not the case in the non-Bayesian setting. For that reason the model class is restricted to linear regression models with deterministic regressors in the non-Bayesian case. Algorithms are proposed for drawing samples of the parameters such that the uncertainties in the system identification models are reflected in the samples. Theoretical results are given bounding the probability that the found solution to the scenario optimisation problem will give a worse performance when applied to the true system than seen on the drawn scenarios. The methods are illustrated in simulation examples showing good performance.

An investigation of optimal sensor locations for multi-type sensors considering modeling uncertainties by Bayesian system identification

This study investigates optimal sensor locations using multi-type sensors for strain recovery. Weighted least squares estimation is used for strain recovery, and the optimal sensor locations are evaluated by minimizing the sum of the covariance of modal displacement coordinates. Here, modeling uncertainty is taken into account by using Bayesian system identification. The present study evaluates multiple optimal sensor locations considering modeling uncertainties and quantifying their plausibility using multi-type sensors. First, a 2-D truss is studied by using an exhaustive search. It has been observed that when strain and acceleration data are fused, the sum of the variances of the modal displacement coordinate estimates is not sensitive to the locations of the acceleration sensors and that multiple optimal solutions exist. In order to obtain a unique solution and extract maximum information from acceleration data too, the sum of the variances of the estimates of a quantity defined as the pseudo-modal acceleration coordinates is also used as a performance measure. Further investigation using Bayesian system identification has revealed that the optimal sensor location with maximum plausibility is the same as the one obtained from the nominal model. Next, the methodology is studied using the 3-D truss model of a bridge in Jhansi, India, and a genetic algorithm is used for optimization. It has been observed that the variance of the estimates is a better measure for selecting the optimal sensor locations than the Fisher information because it can detect ill-conditioning. Finally, the methodology is applied to the experimental data obtained from a four-story shear building to gain further insights. Here also, it has been observed that the variance of the estimates rightly predicts the order of the optimal sensor locations, unlike the Fisher information. When the experimental data are used for Bayesian system identification, it has been noticed that all the predicted OSLs correspond to the one predicted by the nominal model.

A Rapid Evaluation Method for the Vertical Condition of Bridge Substructures Using Bayesian System Identification and Output-Only Measurements

A Rapid Evaluation Method for the Vertical Condition of Bridge Substructures Using Bayesian System Identification and Output-Only Measurements

Bayesian system identification and chaotic prediction from data for stochastic Mathieu-van der Pol-Duffing energy harvester

• An improved approximate Bayesian method is proposed. • The unknown parameters of chaotic VEH system are identified by improved approximate Bayesian method. • The 0-1 test is used to judge the chaotic occur based on the correlation of the response data In this paper, the approximate Bayesian computation combines the particle swarm optimization and sequential Monte Carlo methods, which identify the parameters of the Mathieu-van der Pol-Duffing chaotic energy harvester system. Then the proposed method is applied to estimate the coefficients of the chaotic model and the response output paths of the identified coefficients compared with the observed, which verifies the effectiveness of the proposed method. Finally, a partial response sample of the regular and chaotic responses, determined by the maximum Lyapunov exponent, is applied to detect whether chaotic motion occurs in them by a 0-1 test. This paper can provide a reference for data-based parameter identification and chaotic prediction of chaotic vibration energy harvester systems.

Bayesian system identification for structures considering spatial and temporal correlation

Abstract The decreasing cost and improved sensor and monitoring system technology (e.g., fiber optics and strain gauges) have led to more measurements in close proximity to each other. When using such spatially dense measurement data in Bayesian system identification strategies, the correlation in the model prediction error can become significant. The widely adopted assumption of uncorrelated Gaussian error may lead to inaccurate parameter estimation and overconfident predictions, which may lead to suboptimal decisions. This article addresses the challenges of performing Bayesian system identification for structures when large datasets are used, considering both spatial and temporal dependencies in the model uncertainty. We present an approach to efficiently evaluate the log-likelihood function, and we utilize nested sampling to compute the evidence for Bayesian model selection. The approach is first demonstrated on a synthetic case and then applied to a (measured) real-world steel bridge. The results show that the assumption of dependence in the model prediction uncertainties is decisively supported by the data. The proposed developments enable the use of large datasets and accounting for the dependency when performing Bayesian system identification, even when a relatively large number of uncertain parameters is inferred.

Accelerating computations in two-stage Bayesian system identification with Fisher information matrix and eigenvalue sensitivity

Structural system identification aims at identifying the parameters of a theoretical (e.g., finite element) model using measured data of a constructed structure. As a means of extracting relevant information from noisy vibration data, one common strategy is to adopt a ‘two-stage’ approach, where natural frequencies and mode shapes are first estimated from Stage I (modal identification), and then used for further estimating the structural model parameters (e.g., stiffness, mass) in Stage II. In a Bayesian statistical context with sufficient data so that model parameters can be identified uniquely, the identification result is characterized by the ‘most probable value’ (MPV) that gives the best estimate, and a covariance matrix that quantifies the remaining uncertainty. Computationally, determining the MPV involves solving an optimization problem with a measure-of-fit function; the covariance matrix involves taking the Hessian of the function. Exploiting the mathematical structure of the measure-of-fit function in the two-stage problem, this work proposes a Fisher scoring method for determining the MPV, where the Hessian matrix in a Newton’s iteration is replaced by the Fisher information matrix to eliminate repeated computations of second order derivatives and to improve convergence robustness. Advanced techniques of eigenvalue sensitivity are applied so that the gradient and Hessian involved in MPV and covariance matrix computations can be obtained accurately and efficiently by solving full-rank matrix equations for the subject modes only. The proposed methodology is investigated with synthetic and laboratory experimental data. Performance in terms of convergence robustness and efficiency is compared with existing methods, including simplex search and Newton’s method.

Analysis of Structural Health Monitoring Data with Correlated Measurement Error by Bayesian System Identification: Theory and Application.

Measurement error is non-negligible and crucial in SHM data analysis. In many applications of SHM, measurement errors are statistically correlated in space and/or in time for data from sensor networks. Existing works solely consider spatial correlation for measurement error. When both spatial and temporal correlation are considered simultaneously, the existing works collapse, as they do not possess a suitable form describing spatially and temporally correlated measurement error. In order to tackle this burden, this paper generalizes the form of correlated measurement error from spatial correlation only or temporal correlation only to spatial-temporal correlation. A new form of spatial-temporal correlation and the corresponding likelihood function are proposed, and multiple candidate model classes for the measurement error are constructed, including no correlation, spatial correlation, temporal correlation, and the proposed spatial-temporal correlation. Bayesian system identification is conducted to achieve not only the posterior probability density function (PDF) for the model parameters, but also the posterior probability of each candidate model class for selecting the most suitable/plausible model class for the measurement error. Examples are presented with applications to model updating and modal frequency prediction under varying environmental conditions, ensuring the necessity of considering correlated measurement error and the capability of the proposed Bayesian system identification in the uncertainty quantification at the parameter and model levels.

Bayesian System Identification of Rail–Sleeper–Ballast System in Time and Modal Domains: Comparative Study

Bayesian System Identification of Rail–Sleeper–Ballast System in Time and Modal Domains: Comparative Study

Bayesian parameter updating in linear structural dynamics with frequency transformed data using rational surrogate models

The identification of parameters of structural models through measurements of the system’s response is of interest in many contexts. In Bayesian system identification the underlying inverse problem is formulated in a probabilistic setting, and Bayes’ rule is applied to update a prior conjecture on the parameters. We apply the Bayesian framework to identify the parameters of structural systems using dynamic measurement data. The likelihood function is formulated in terms of the misfit of the frequency transformed data and the model frequency response function. We introduce a novel formulation that accounts for the correlation of the model error in both spatial and frequency domain. The proposed formulation is able to handle dense data sets in the frequency domain without need to manually select data points. Due to the high computational demands of sampling-based approaches for solving the Bayesian updating problem with expensive structural dynamics models, we resort to surrogate models. We apply a recently introduced rational surrogate model that approximates the complex frequency response as a rational of two polynomials with complex coefficients. Samples of the posterior distribution of the model parameters are then obtained through an adaptive sequential sampling approach using the surrogate instead of the original dynamic model. The proposed method is successfully applied to identify the orthotropic stiffness and damping parameters of a finite element model of a cross laminated timber plate.

An Overcomplete Approach to Fitting Drift-Diffusion Decision Models to Trial-By-Trial Data.

Drift-diffusion models or DDMs are becoming a standard in the field of computational neuroscience. They extend models from signal detection theory by proposing a simple mechanistic explanation for the observed relationship between decision outcomes and reaction times (RT). In brief, they assume that decisions are triggered once the accumulated evidence in favor of a particular alternative option has reached a predefined threshold. Fitting a DDM to empirical data then allows one to interpret observed group or condition differences in terms of a change in the underlying model parameters. However, current approaches only yield reliable parameter estimates in specific situations (c.f. fixed drift rates vs drift rates varying over trials). In addition, they become computationally unfeasible when more general DDM variants are considered (e.g., with collapsing bounds). In this note, we propose a fast and efficient approach to parameter estimation that relies on fitting a “self-consistency” equation that RT fulfill under the DDM. This effectively bypasses the computational bottleneck of standard DDM parameter estimation approaches, at the cost of estimating the trial-specific neural noise variables that perturb the underlying evidence accumulation process. For the purpose of behavioral data analysis, these act as nuisance variables and render the model “overcomplete,” which is finessed using a variational Bayesian system identification scheme. However, for the purpose of neural data analysis, estimates of neural noise perturbation terms are a desirable (and unique) feature of the approach. Using numerical simulations, we show that this “overcomplete” approach matches the performance of current parameter estimation approaches for simple DDM variants, and outperforms them for more complex DDM variants. Finally, we demonstrate the added-value of the approach, when applied to a recent value-based decision making experiment.

Bayesian positive system identification: Truncated Gaussian prior and hyperparameter estimation

Bayesian methods have been extended for the linear system identification problem in the past ten years. The traditional Bayesian identification selects a Gaussian prior and considers the tuning of kernels, i.e., the covariance matrix of a Gaussian prior. However, Gaussian priors cannot express the system information appropriately for identifying a positive finite impulse response (FIR) model. This paper exploits the truncated Gaussian prior and develops Bayesian identification procedures for positive FIR models. The proposed parameterizations in the truncated Gaussian prior can reflect the decay rate and the correlation of the impulse response of the system to be identified. The expectation–maximization (EM) algorithm is tailored to the hyperparameter estimation problem of positive system identification with the truncated Gaussian prior. Numerical experiments compare the truncated Gaussian prior to the traditional Gaussian prior for positive FIR system identification. The simulation results demonstrate that the truncated Gaussian prior outperforms the Gaussian prior.

Bayesian model updating-based and simulation-based system identification of shear-bending model for high-rise building

Bayesian model updating-based and simulation-based system identification of shear-bending model for high-rise building

Time-domain Markov chain Monte Carlo–based Bayesian damage detection of ballasted tracks using nonlinear ballast stiffness model

This article reports the development of a methodology for detecting ballast damage under a sleeper based on measured sleeper vibration following the Bayesian statistical system identification framework. To ensure the methodology is applicable under large amplitude vibration of the sleeper (e.g. under trainload), the nonlinear stress–strain behavior of railway ballast is considered. This, on one hand, significantly reduces the problem of modeling error, but, on the other hand, increases the number of uncertain model parameters. The uncertainty associated with the identified model parameters of the rail–sleeper–ballast system may be very high. To overcome this difficulty, the Markov chain Monte Carlo–based Bayesian model updating is adopted in the proposed methodology for the approximation of the posterior probability density function of uncertain model parameters. Owing to the nonlinear behavior of the system, the model updating is performed in the time domain instead of the modal domain. The applicability of the proposed damage detection methodology was first verified numerically using simulated impact hammer test data in two damaged cases perturbed with Gaussian white noise. Second, impact hammer tests of in situ sleepers in the full-scale in-door ballasted track test panel were carried out to collect data for the experimental verification of the proposed methodology. Artificial ballast damage was simulated under the target concrete sleeper by replacing normal-sized ballast particles (∼60 mm) by small-sized ballast particles (∼15 mm). The proposed methodology successfully identified the location and severity of ballast damage under the sleeper. From the calculated posterior marginal probability density functions of model parameters, one can quantify the uncertainties associated with the damage detection results. The proposed methodology is an essential step in the development of a long-term railway track health monitoring system utilizing train-induced vibration.

Novel sparseness-inducing dual Kalman filter and its application to tracking time-varying spatially-sparse structural stiffness changes and inputs

While the theory of Bayesian system identification provides a probabilistic means for reliably and robustly inferring models of a dynamic system and their parameters based on measured dynamic response, exploiting sparseness during online tracking of changing model parameters is not well understood. The focus in this study is to implement the dual Kalman filter for real-time Bayesian sequential state and parameter identification based on noisy sensor signals while incorporating sparse Bayesian learning to impose sparse model parameter changes from their initial reference values. We also want out model to be able to capture the evolution of sparse model parameter changes between two successive time instants. To this end, we present a hierarchical Bayesian model for tracking the joint posterior distribution of the state and model parameter vectors for a monitored dynamical system, where the two afore-mentioned sparseness constraints (sparse changes from reference values and with time) are also effectively incorporated for each time. We show our stochastic model of the structural dynamical system can be represented as a coupled conditionally-linear Gaussian state space model for the state and model parameter vectors, leading to some interesting analytical properties of the method that allow quantities of interest to be calculated in real time by using Kalman filtering equations. The parameters for the measurement and state prediction errors are learned solely from the available data up to the current time and so our method resolves the well-known instability problem in Kalman filtering due to arbitrary assignment of the error-distribution parameters. Finally, two illustrative applications are presented, one for identification of stiffness degradation and the other for input time–history identification where both are based on noisy dynamic response measurements from a structure.

Bayesian system ID: optimal management of parameter, model, and measurement uncertainty

We evaluate the robustness of a probabilistic formulation of system identification (ID) to sparse, noisy, and indirect data. Specifically, we compare estimators of future system behavior derived from the Bayesian posterior of a learning problem to several commonly used least squares-based optimization objectives used in system ID. Our comparisons indicate that the log posterior has improved geometric properties compared with the objective function surfaces of traditional methods that include differentially constrained least squares and least squares reconstructions of discrete time steppers like dynamic mode decomposition (DMD). These properties allow it to be both more sensitive to new data and less affected by multiple minima --- overall yielding a more robust approach. Our theoretical results indicate that least squares and regularized least squares methods like dynamic mode decomposition and sparse identification of nonlinear dynamics (SINDy) can be derived from the probabilistic formulation by assuming noiseless measurements. We also analyze the computational complexity of a Gaussian filter-based approximate marginal Markov Chain Monte Carlo scheme that we use to obtain the Bayesian posterior for both linear and nonlinear problems. We then empirically demonstrate that obtaining the marginal posterior of the parameter dynamics and making predictions by extracting optimal estimators (e.g., mean, median, mode) yields orders of magnitude improvement over the aforementioned approaches. We attribute this performance to the fact that the Bayesian approach captures parameter, model, and measurement uncertainties, whereas the other methods typically neglect at least one type of uncertainty.

Two-stage Bayesian system identification using Gaussian discrepancy model

System identification aims at updating the model parameters (e.g. mass and stiffness) associated with the mathematical model of a structure based on measured structural response. In this process, a two-stage approach is commonly adopted. In Stage I, modal parameters including natural frequencies and mode shapes are identified. In Stage II, the modal parameters are used to update structural parameters such as those related to stiffness, mass, and boundary conditions. A recent Bayesian formulation allows the identification results in the first stage to be incorporated in the second stage directly via Bayes’ rule without using a heuristic model (often based on classical statistics) that transfers the information from Stages I to II. This opens up opportunities for explicitly accounting for modeling error in the structural model (Stage II) through the conditional distribution of modal parameters given structural model parameters. Following this approach, this article investigates a methodology where the modeling error between the two stages is incorporated with Gaussian distributions whose statistical parameters are also updated with available data. Leveraging on special mathematical structure induced by the model, computational issues are resolved and an analytical investigation is performed that yields insights on the role of modeling error and whether its statistics can be distinguished from those of identification uncertainty (defined for given structural model). The proposed methodology is verified using synthetic data and applied to a laboratory-scale structure.

A stochastic framework for hydraulic performance assessment of complex water distribution networks: Application to connectivity detection problems

This paper is concerned with the hydraulic performance assessment of large scale water distribution networks in presence of uncertainty. In particular, the associate connectivity detection problem is examined in detail. For this purpose, a Bayesian system identification methodology is combined with an efficient hydraulic simulation model. A number of hydraulic model classes are defined as potential connectivity events. Based on information from flow rates in the pipes, the proposed updating technique provides estimates of the most probable connectivity scenarios. Such scenarios correspond to the model classes that maximize their evidences or posterior probabilities. The effectiveness of the proposed identification framework is illustrated by applying the connectivity detection approach to a real water distribution system.

Bayesian inference for damage identification based on analytical probabilistic model of scattering coefficient estimators and ultrafast wave scattering simulation scheme

Ultrasonic Guided Waves (GW) actuated by piezoelectric transducers installed on structures have proven to be sensitive to small structural defects, with acquired scattering signatures being dependent on the damage type. This study presents a generic framework for probabilistic damage characterization within complex structures, based on physics-rich information on ultrasound wave interaction with existent damage. To this end, the probabilistic model of wave scattering properties estimated from measured GWs is inferred based on absolute complex-valued ratio statistics. Based on the probabilistic model, the likelihood function connecting the scattering properties predicted by a computational model containing the damage parametric description and the scattering estimates is formulated within a Bayesian system identification framework to account for measurement noise and modelling errors. The Transitional Monte Carlo Markov Chain (TMCMC) is finally employed to sample the posterior probability density function of the updated parameters. However, the solution of a Bayesian inference problem often requires repeated runs of “expensive-to-evaluate” Finite Element (FE) simulations, making the inversion procedure firmly demanding in terms of runtime and computational resources. To overcome the computational challenges of repeated likelihood evaluations, a cheap and fast Kriging surrogate model built and based on a set of training points generated with an experiment design strategy in tandem with a hybrid Wave and Finite Element (WFE) computational scheme is proposed in this study. In each “numerical experiment”, the training outputs (i.e. ultrasound scattering properties) are efficiently computed using the hybrid WFE scheme which combines conventional FE analysis with periodic structure theory. By establishing the relationship between the training outputs and damage characterization parameters statistically, the surrogate model further enhances the computational efficiency of the exhibited scheme. Two case studies including one numerical example and an experimental one are presented to verify the accuracy and efficiency of the proposed algorithm.

Measurement of train-induced vibration for track parameter identification: Bayesian probabilistic approach

The train-induced vibration response is used to develop a comprehensive finite element model for the railway track, which considered rail-sleeper-ballast interaction. This model can be further use in the development of an operational monitoring system for railway track by following the Bayesian statistical system identification framework. However, it is believed that the uncertainties associated with the measured train-induced test data and model parameters are relatively high, therefore, the corresponding model updating problem will very likely to be in the category of unidentifiable case. Therefore, Markov chain Monte Carlo (MCMC) simulation was used in generating the samples for the approximation of the posterior uncertainties. The measurements also provide not only reference values suitable for model fitting, but also a good insight into the main features of the dynamic behaviour of the railway track system. Finally, the reliability and accuracy of the proposed method are demonstrated by numerically reproducing the system responses. Modelling assumptions are discussed, together with their implications on numerical results of the time-domain analysis, which were found to be in good agreement with experimental results.