Abstract
In this paper, we continue our efforts to show how maximum relative entropy (MrE) can be used as a universal updating algorithm. Here, our purpose is to tackle a joint state and parameter estimation problem where our system is nonlinear and in a non-equilibrium state, i.e., perturbed by varying external forces. Traditional parameter estimation can be performed by using filters, such as the extended Kalman filter (EKF). However, as shown with a toy example of a system with first order non-homogeneous ordinary differential equations, assumptions made by the EKF algorithm (such as the Markov assumption) may not be valid. The problem can be solved with exponential smoothing, e.g., exponentially weighted moving average (EWMA). Although this has been shown to produce acceptable filtering results in real exponential systems, it still cannot simultaneously estimate both the state and its parameters and has its own assumptions that are not always valid, for example when jump discontinuities exist. We show that by applying MrE as a filter, we can not only develop the closed form solutions, but we can also infer the parameters of the differential equation simultaneously with the means. This is useful in real, physical systems, where we want to not only filter the noise from our measurements, but we also want to simultaneously infer the parameters of the dynamics of a nonlinear and non-equilibrium system. Although there were many assumptions made throughout the paper to illustrate that EKF and exponential smoothing are special cases ofMrE, we are not “constrained”, by these assumptions. In other words, MrE is completely general and can be used in broader ways.
Highlights
In this paper, we continue our efforts to show how maximum relative entropy (MrE) [1] links to existing data filtering and parameter estimation approaches in data analysis
In our previous paper [2], we showed the direct connection between MrE and the Kalman filter (KF) [3], which resulted in KF being shown as a special case of MrE
This demonstrates that an exponential smoothing filter is a special case of using MrE as a filter with first order nonhomogenous ordinary differential constraints applied to the posterior solution
Summary
We continue our efforts to show how maximum relative entropy (MrE) [1] links to existing data filtering and parameter estimation approaches in data analysis. We extend the data assimilation approach by performing the data assimilation (calculating estimates), and inferring the parameters of our mathematical model by incorporating additional information about the real control signal. In many systems, such as weather and climate, imposing a control signal is not possible. The main purpose of this paper is to show how MrE can be used to simultaneously infer both the parameters and state values, while taking into account the transient characteristic of the exponential process This demonstrates that an exponential smoothing filter is a special case of using MrE as a filter with first order nonhomogenous ordinary differential constraints applied to the posterior solution
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have