Wonham Filter Research Articles

Analysis of Algorithms of Numerical Implementations for the Wonham Filter Under Uncertainty in Measurements Noise Covariance

The paper addresses the filtering a continuous-time Markov chain states that can be observed through linear measurements perturbed by a Wiener process. There is supposed the presence of uncertainty in the intensity of measurements noises. The problem is worked out under the assumption of unknown intensity but subject to its known upper bound. If there is no uncertainty in measurements the optimal solution is provided by the Wonham filter that doesn't ensure stable numerical implementations. The paper exposes that the Wonham filter shows robustness in the presence of uncertainty if model's parameters don't imply its divergent. It is detected that to cope with divergence tracking and handling trajectories aren't sufficient in the case of uncertainty. The more efficient way is to consider discretized approximations of the Wonham filter implemented for a discrete model that approximates the initial continuous-time measurements system. Such an approach perceptibly advantages if numerical implementations contain divergent trajectories. If there are no divergent trajectories, then the discretized filters give a slightly worse result but acceptable.

Applications of the Linear Quadratic Regulator Optimal Control With Completely and Partially Observed Markovian Switching

Abstract We study optimal control problems where the dynamic system evolves according to a linear stochastic differential equation with (a) multiplicative and additive white noise, (b) a mixture of white noise, and (c) white noise and colored noise. The drift rate and the diffusion matrix of the linear dynamic system depend on a continuous-time Markov chain with finite state space that is (I) partially observed and (II) completely observed. Using the results of Wonham filter theory, we reduce the partially observed problems to one with complete observation. We solve the control optimal problems explicitly with the help of dynamic programing technique, and three applications are presented to illustrate our theoretical results.

Comparative Study of Markov Chain Filtering Schemas for Stabilization of Stochastic Systems under Incomplete Information

The object under investigation is a controllable linear stochastic differential system affected by some external statistically uncertain piecewise continuous disturbances. They are directly unobservable but assumed to be a continuous-time Markov chain. The problem is to stabilize the system output concerning a quadratic optimality criterion. As is known, the separation theorem holds for the system. The goal of the paper is performance analysis of various numerical schemes applied to the filtering of the external Markov input for system stabilization purposes. The paper briefly presents the theoretical solution to the considered problem of optimal stabilization for systems with the Markov jump external disturbances: the conditions providing the separation theorem, the equations of optimal control, and the ones defining the Wonham filter. It also contains a complex of the stable numerical approximations of the filter, designed for the time-discretized observations, along with their accuracy characteristics. The approximations of orders 12, 1, and 2 along with the classical Euler–Maruyama scheme are chosen for the comparison of the Wonham filter numerical realization. The filtering estimates are used in the practical stabilization of the various linear systems of the second order. The numerical experiments confirm the significant influence of the filtering precision on the stabilization performance and superiority of the proposed stable schemes of numerical filtering.

A Logarithmic Bayesian Approach to Quantum Error Detection

We consider the problem of continuous quantum error correction from a Bayesian perspective, proposing a pair of digital filters using logarithmic probabilities that are able to achieve near-optimal performance on a three-qubit bit-flip code, while still being reasonable to implement on low-latency hardware. These practical filters are approximations of an optimal filter that we derive explicitly for finite time steps, in contrast with previous work that has relied on stochastic differential equations such as the Wonham filter. By utilizing logarithmic probabilities, we are able to eliminate the need for explicit normalization and can reduce the Gaussian noise distribution to a simple quadratic expression. The state transitions induced by the bit-flip errors are modeled using a Markov chain, which for log-probabilties must be evaluated using a LogSumExp function. We develop the two versions of our filter by constraining this LogSumExp to have either one or two inputs, which favors either simplicity or accuracy, respectively. Using simulated data, we demonstrate that the single-term and two-term filters are able to significantly outperform both a double threshold scheme and a linearized version of the Wonham filter in tests of error detection under a wide variety of error rates and time steps.

Optimal Stabilization of Linear Stochastic System with Statistically Uncertain Piecewise Constant Drift

The paper presents an optimal control problem for the partially observable stochastic differential system driven by an external Markov jump process. The available controlled observations are indirect and corrupted by some Wiener noise. The goal is to optimize a linear function of the state (output) given a general quadratic criterion. The separation principle, verified for the system at hand, allows examination of the control problem apart from the filter optimization. The solution to the latter problem is provided by the Wonham filter. The solution to the former control problem is obtained by formulating an equivalent control problem with a linear drift/nonlinear diffusion stochastic process and with complete information. This problem, in turn, is immediately solved by the application of the dynamic programming method. The applicability of the obtained theoretical results is illustrated by a numerical example, where an optimal amplification/stabilization problem is solved for an unstable externally controlled step-wise mechanical actuator.

Exponential contraction of switching jump diffusions with a hidden Markov chain

This paper is concerned with the exponential contraction of Markovian switching jump diffusions that involve a partially observed Markov chain. For such partially observed problems, we use a Wonham filter to estimate the Markov chain from the observable evolution of the given process, and convert the original system to a completely observable one. We then establish sufficient conditions of feedback controls for exponential contraction in mean square and almost sure exponential contraction.

Continuous quantum error correction for evolution under time-dependent Hamiltonians

We develop a protocol for continuous operation of a quantum error correcting code for protection of coherent evolution due to an encoded Hamiltonian against environmental errors, using the three-qubit bit-flip code and bit-flip errors as a canonical example. To detect errors in real time, we filter the output signals from continuous measurement of the error syndrome operators and use a double thresholding protocol for error diagnosis, while correction of errors is done as in the conventional operation. We optimize our continuous operation protocol for evolution under quantum memory and under quantum annealing, by maximizing the fidelity between the target and actual logical states at a specified final time. In the case of quantum memory, we show that our continuous operation protocol yields a logical error rate that is slightly larger than the one obtained from using the optimal Wonham filter for error diagnosis while being simpler to implement. For quantum annealing, we show that our continuous quantum error correction protocol can significantly reduce the final logical state infidelity when the continuous measurements are sufficiently strong relative to the strength of the time-dependent Hamiltonian. We also show that this continuous quantum error correction protocol can reduce the time-to-solution relative to the value obtained from a classical parallelization scheme. These results suggest that a continuous implementation is suitable for quantum error correction in the presence of encoded time-dependent Hamiltonians, opening the possibility of many applications in quantum simulation and quantum annealing.

Optimal exploitation for hybrid systems of renewable resources under partial observation

This work focuses on optimal controls for hybrid systems of renewable resources in random environments. We propose a new formulation to treat the optimal exploitation with harvesting and renewing. The random environments are modeled by a Markov chain, which is hidden and can be observed only in a Gaussian white noise. We use the Wonham filter to estimate the state of the Markov chain from the observable process. Then we formulate a harvesting–renewing model under partial observation. The Markov chain approximation method is used to find a numerical approximation of the value function and optimal policies. Our work takes into account natural aspects of the resource exploitation in practice: interacting resources, switching environment, renewing and partial observation. Numerical examples are provided to demonstrate the results and explore new phenomena arising from new features in the proposed model.

Always-On Quantum Error Tracking with Continuous Parity Measurements

We investigate quantum error correction using continuous parity measurements to correct bit-flip errors with the three-qubit code. Continuous monitoring of errors brings the benefit of a continuous stream of information, which facilitates passive error tracking in real time. It reduces overhead from the standard gate-based approach that periodically entangles and measures additional ancilla qubits. However, the noisy analog signals from continuous parity measurements mandate more complicated signal processing to interpret syndromes accurately. We analyze the performance of several practical filtering methods for continuous error correction and demonstrate that they are viable alternatives to the standard ancilla-based approach. As an optimal filter, we discuss an unnormalized (linear) Bayesian filter, with improved computational efficiency compared to the related Wonham filter introduced by Mabuchi [New J. Phys. 11, 105044 (2009)]. We compare this optimal continuous filter to two practical variations of the simplest periodic boxcar-averaging-and-thresholding filter, targeting real-time hardware implementations with low-latency circuitry. As variations, we introduce a non-Markovian ``half-boxcar'' filter and a Markovian filter with a second adjustable threshold; these filters eliminate the dominant source of error in the boxcar filter, and compare favorably to the optimal filter. For each filter, we derive analytic results for the decay in average fidelity and verify them with numerical simulations.

Optimal Retirement Problem Under Partial Information

The optimal retirement decision is an optimal stopping problem when retirement is irreversible. We investigate the optimal consumption, investment, and retirement decisions when the mean return of a risky asset is unobservable and is estimated by filtering from historical prices. To ensure non-negativity of the consumption rate and the borrowing constraints on the wealth process of the representative agent, we conduct our analysis using a duality approach. We link the dual problem to American option pricing with stochastic volatility and prove that the duality gap is closed. We then apply our theory to a hidden Markov model for regime-switching mean return with Bayesian learning or the Wonham filter. We fully characterize the existence and uniqueness of variational inequality in the dual optimal stopping problem, as well as the free boundary of the problem. An asymptotic closed-form solution is derived for optimal retirement timing by small-scale perturbation. We discuss the potential applications of the results to other partial information settings.

Feedback controls for desired dynamical behaviours of a new stochastic ecosystem with regime switching

This paper is concerned with the feedback control problems of a new stochastic ecosystem with regime switching expressed by a partially observable Markov chain. The main idea is to convert the partially observable system into a complete observable one by estimating the unobservable Markov chain by Wonham filter. For the complete observable system, the authors discuss the existence, uniqueness, stochastic boundedness and sample path continuity of the positive solution by constructing Lyapunov functions, and design various forms of feedback controls to regulate the long-time dynamic behaviours of the system. The idea and methods in this paper can also be applied to other partially observable control systems such as financial systems, infections disease system and so on.

Optimal linear mean square filter for the operation mode of continuous‐time Markovian jump linear systems

This paper makes a further foray on the study of the filtering problem for the class of Markov jump linear systems (MJLSs). The authors shall be particularly interested in the filtering problem for the Markov jump parameter (the operation mode). Previous result in the literature on this problem has been obtained by Wonham, which has derived an optimal non-linear filter for this problem. The main hindrance with Wonham's result, in the context of the optimal control problem for MJLS with partial observation of the operation mode, is that it introduces a great deal of non-linearity in the Hamilton-Jacobi-Belman equation, which makes it difficult to get an explicit closed solution for the control problem. Motivated, in part, by this, the main contribution of this paper is to devise an optimal linear filter for the mode operation, which they believe could be more favourable in the solution of the control problem with partial observations. In addition, relying on Murayama's stochastic numerical method and the results by Chenggui-Yuan, they carry out simulation of Wonham's filter, and the one devised in this paper, in order to compare their performances.

Block Trading: Building up a Stock Position Under a Regime Switching Model

This work is intended to systematically characterize the problem of block trading. The author characterizes it as a constrained optimal purchasing (or control) problem, on behalf of the vendee. The purpose of this work is to investigate the optimal purchasing strategies and give quantitative reference information on the pricing of block trade. Noting that large block of stock trading may result in the price fluctuation and different market environment may lead to distinct investing strategies, the controlled diffusion model with regime switching is adopted, which can be used to represent distinct environment of financial market, and the model is modified to allow the price impact, by allowing the drift depend on the purchase rate. Where the switching regimes can be completely observable or unobservable (hidden). Which is a significant difference in contrast to most market models in the existing literature. The objective is to maximize the total discounted shares of the underlying stock, subject to the expected fund for building up corresponding position with an upper bound. This work mainly focuses on the completely observable case, as to the case with hidden regimes, this work just briefly talk about how to reduce the unobservable model to a completely observable one. In the completely observable case, to solve this constrained control problem, the Lagrange multiplier methods and the dynamic programming approaches are used. Though optimality is obtained, what is still lacking is an explicit solution to the optimality equation. To overcome this difficulty, the two loop approximation scheme is given. For fixed Lagrange multiplier, the inner loop is to obtain optimal strategies, which is based mainly on the finite difference approximation. While, the outer loop is a stochastic recursive approximation algorithm to get the optimal Lagrange multiplier. Convergence results are obtained for both the inner and the outer approximations. The quantitative pricing information is related to the threshold levels of the optimal purchasing strategy. Numerical examples are also provided to illustrate our results. Finally, as to the unobservable case, by using the well known Wonham filter approach, one can reduce the unobservable case to a completely observable one.

Optimal dividend policy with liability constraint under a hidden Markov regime-switching model

This paper deals with the optimal liability and dividend strategies for an insurance company in Markov regime-switching models. The objective is to maximize the total expected discounted utility of dividend payment in the infinite time horizon in the logarithm and power utility cases, respectively. The switching process, which is interpreted by a hidden Markov chain, is not completely observable. By using the technique of the Wonham filter, the partially observed system is converted to a completely observed one and the necessary information is recovered. The upper-lower solution method is used to show the existence of classical solution of the associated second-order nonlinear Hamilton-Jacobi-Bellman equation in the two-regime case. The explicit solution of the value function is derived and the corresponding optimal dividend policies and liability ratios are obtained. In the multi-regime case, a general setting of the Wonham filter is presented, and the value function is proved to be a viscosity solution of the associated system of Hamilton-Jacobi-Bellman equations.

Dissipative control of a three-species food chain stochastic system with a hidden Markov chain

This paper focuses on a three-species food chain system which is formulated as stochastic differential equations with regime switching represented by a hidden Markov chain. Firstly, using the Wonham filter, we estimate the hidden Markov chain through the observable solution of the Markov chain in Gaussian white noise. Then two kinds of special dissipative control strategy are proposed to study the given model. That is, under H_{infty} control and passive control, the sufficient conditions for global asymptotic stability are established, respectively. Finally, numerical examples are given to illustrate the effectiveness of the theoretical results.

A Nash equilibrium filter

ABSTRACTThe Wonham filter, which estimates a Markov chain observed in Brownian noise, is considered. However, the parameters of the observation process are not known. Maximizing the un-normalized probabilities of the Zakai equation over the parameters leads to a Nash equilibrium whose solution is discussed using the stochastic control results of Peng and Yong and Zhou.

Numerical methods for optimal harvesting strategies in random environments under partial observations

This work is concerned with optimal harvesting problems in random environments. In contrast to the existing literature, the Markov chain is hidden and can only be observed in a Gaussian white noise in our work. We first use the Wonham filter to estimate the Markov chain from the observable evolution of the given process so as to convert the original problem to a completely observable one. Then we treat the resulting optimal control problem. Because the problem is virtually impossible to solve in closed form, our main effort is devoted to developing numerical approximation algorithms. To approximate the value function and optimal strategies, Markov chain approximation methods are used to construct a discrete-time controlled Markov chain. Convergence of the algorithm is proved by weak convergence method and suitable scaling. A numerical example is provided to demonstrate the results.

Feedback Particle Filter for a Continuous-time Markov Chain

This technical note extends the feedback particle filter (FPF) methodology and algorithms to the problem of filtering a continuous-time Markov chain. The main contribution is the development of a feedback control-based transformation of the Wonham filter, where the control input is realized via a time-modulated Poisson counter process. A complete characterization of the feedback mechanism that defines the FPF is obtained, which leads to tractable algorithms for the nonlinear filtering problem even for large state spaces. Numerical examples are introduced to help illustrate the application of these techniques.

Mean-variance type controls involving a hidden Markov chain: models and numerical approximation

Motivated by applications arising in networked systems, this work examines controlled regime-switching systems that stem from a mean-variance formulation. A main point is that the switching process is a hidden Markov chain. An additional piece of information, namely, a noisy observation of switching process corrupted by white noise is available. We focus on minimizing the variance subject to a fixed terminal expectation. Using the Wonham filter, we convert the partially observed system to a completely observable one first. Since closed-form solutions are virtually impossible be obtained, a Markov chain approximation method is used to devise a computational scheme. Convergence of the algorithm is obtained. A numerical example is provided to demonstrate the results.

Probabilistic Market Timing Using Bull Bear Cycle Statistics

Under the historical market view of binary bull and bear cycles, what is an ex ante optimal trading strategy? Similar to an original optimal stopping time model (Dai, et al 2010, 2011) to maximize long term investment returns, we introduce a market timing strategy based on the probability that the market is currently in a bull or bear regime. In doing so, we adjust the Wonham filter (1965) for the regime conditional probability calculation to allow time-varying volatilities and utilize expected bull/bear regime risk-adjusted average returns as statistically estimated parameter inputs. Further, we derive explicit formula to approximate the optimal probabilistic buying or selling thresholds and extend the analytical formulation to allow both long only and long/flat/short positions for a hysteresis trading implementation. Considering the fractal characteristics of stock market indexes and other asset class index data at different frequencies, we devise a systematic procedure to separate historical bull bear regimes and obtain detailed market cycle statistics. This allows using different index gain, loss and duration thresholds to define historical bull/bear cycles in the same manner as an earlier study (Wagner, 1992). We back test the methodology on two broad stock indexes: S&P 500 Index (SPX 1928-2014) and Shanghai Stock Exchange Composite Index (SSEC 1995-2014). The results, including that of an out-of-sample test case on the SPX since 1940 using 10-cycle look-back rolling statistics, indicate out-performance over the buy-and-hold approach and a popular trend following technical trading strategy, the Golden Cross.