Independent Markov Chains Research Articles

Reply to Comment on “Improving Bayesian Model Averaging for Ensemble Flood Modeling Using Multiple Markov Chains Monte Carlo Sampling” by Jasper Vrugt

AbstractThis discussion is a reply to the comments made by Dr. Jasper Vrugt on the Metropolis‐Hastings (M‐H) algorithm with multiple independent Markov chains proposed by Huang and Merwade (2023a), https://doi.org/10.1029/2023wr034947 concerning the validity of the methodology in estimating Bayesian model averaging (BMA) parameters (weights and variances) of the framework proposed by Raftery et al. (2005), https://doi.org/10.1175/mwr2906.1. In this reply, we address his concerns by emphasizing the motivation of applying the proposed M‐H algorithm to BMA analysis and the applicability of the effective sample size that accounts for the autocorrelation across samples in evaluating the efficiency of Markov chain Monte Carlo sampling. Moreover, the details of sampling procedure for BMA prediction distribution are clarified. On the other hand, we present a fair comparison of the default Expectation‐Maximization, M‐H, and differential evolution adaptive Metropolis (DREAM) algorithms in estimating BMA parameters based on a numerical experiment. Results reinforce the findings obtained from Huang and Merwade (2023a) https://doi.org/10.1029/2023wr034947 and further indicate that the proposed M‐H algorithm is better than the DREAM algorithm in terms of sampling efficiency and prediction accuracy. Accordingly, we raise concerns on the use of DREAM algorithm in BMA analysis and suggest conducting peer reviews on the MODELAVG toolbox.

Characterization of geological uncertainties from limited boreholes using copula-based coupled Markov chains for underground construction

This paper proposes an efficient method for quantifying the stratigraphic uncertainties and modeling the geological formations based on boreholes. Two Markov chains are used to describe the soil transitions along different directions, and the transition probability matrices (TPMs) of the Markov chains are analytically expressed by copulas. This copula expression is efficient since it can represent a large TPM by a few unknown parameters. Due to the analytical expression of the TPMs, the likelihood function of the Markov chain model is given in an explicit form. The estimation of the TPMs is then re-casted as a multi-objective constrained optimization problem that aims to maximize the likelihoods of two independent Markov chains subject to a set of parameter constraints. Unlike the method which determines the TPMs by counting the number of transitions between soil types, the proposed method is more statistically sound. Moreover, a random path sampling method is presented to avoid the directional effect problem in simulations. The soil type at a location is inferred from its nearest known neighbors along the cardinal directions. A general form of the conditional probability, based on Pickard's theorem and Bayes rule, is presented for the soil type generation. The proposed stratigraphic characterization and simulation method is applied to real borehole data collected from a construction site in Wuhan, China. It is illustrated that the proposed method is accurate in prediction and does not show an inclination during simulation.

Improving Bayesian Model Averaging for Ensemble Flood Modeling Using Multiple Markov Chains Monte Carlo Sampling

AbstractAs all kinds of numerical models are emerging in hydrologic and hydraulic engineering, Bayesian model averaging (BMA) is one of the popular multi‐model methods used to account for various uncertainties in the flood modeling process and generate robust ensemble predictions. The reliability of BMA parameters (weights and variances) determines the accuracy of BMA predictions. However, the uncertainty in BMA parameters with fixed values, which are usually obtained from Expectation‐Maximization (EM) algorithm, has not been adequately investigated in BMA‐related applications over the past few decades. Given the limitations of the default EM algorithm, Metropolis‐Hastings (M‐H) algorithm, which is one of the most widely used algorithms in Markov Chain Monte Carlo (MCMC), is proposed to estimate BMA parameters. Both numerical experiments and one‐dimensional Hydrologic Engineering Center‐River Analysis System models are employed to examine the applicability of M‐H algorithm with multiple independent Markov chains. The performances of EM and M‐H algorithms are compared based on the daily water stage predictions from 10 model members. Results show that BMA weights estimated from both algorithms are comparable, while BMA variances obtained from M‐H algorithm are closer to the given variances in the numerical experiment. Moreover, the normal proposal distribution used in M‐H algorithm can yield narrower distributions for BMA weights than those from the uniform proposal distribution. Overall, MCMC approach with multiple chains can provide more information associated with the uncertainty of BMA parameters and its performance is better than the default EM algorithm in terms of both deterministic and probabilistic evaluation metrics as well as algorithm flexibility.

Security sliding mode control for networked control system under multi‐channel Markovian hybrid cyber‐attacks

AbstractThis work investigates the security sliding mode control problem of uncertain discrete‐time systems under an unreliable network, in which denial of service and deception attacks may randomly occur in each sensor‐to‐controller channel. Two 2‐valued variables are introduced to model the available state signals for the controller side under random attacks. Meanwhile, by means of mapping rule, independent Markov chains are reformulated into a new Markov chain. And then, a hybrid‐attack‐probability dependent sliding function is constructed, and a desirable sliding mode controller is designed by adopting compensation strategy and available state signals. Moreover, sufficient conditions are derived, under which the closed‐loop system achieve stochastic stability along a quasi sliding mode. Moreover, a cone complementarity linearization (CCL)‐based iterative solving algorithm is proposed. Finally, simulation results are provided to verify the proposed security control approach.

Adaptive neural network control for Markov jumping systems against deception attacks

This paper proposes an innovative approach for mitigating the effects of deception attacks in Markov jumping systems by developing an adaptive neural network control strategy. To address the challenge of dual-mode monitoring mechanisms, two independent Markov chains are used to describe the state changes of the system and the intermittent actuator. By employing a mapping technique, these individual chains are amalgamated into a unified joint Markov chain. Additionally, to effectively approximate the unbounded false signals injected by deception attacks, an adaptive neural network technique is skillfully built. A mode monitoring scheme is implemented to design an asynchronous control law that links the mode information between the joint Markov chain and controller with fewer modes. The paper derives sufficient criteria for the mean-square bounded stability of the resulting system based on Lyapunov theories. Finally, a numerical experiment is conducted to demonstrate the effectiveness of the proposed method.

Stochastic quantized control for memristive neural networks with mixed semi-Markov jump and sampled-data communications using a novel approach

In this paper, the issue of stability and stabilization for semi-Markov jump memristive neural networks (SMJMNNs) with stochastic quantized sampled-data control (QSDC) law is addressed. Firstly, a memristive neural network (MNN) model with mixed semi-Markov jump is established in the framework of the three independent Markov chains. Then, a stochastic QSDC scheme is proposed, in which semi-Markov jump parameters (SMJPs) in the gain matrices, quantization parameters and system connection weight matrices are different from each other. On the other hand, since signal transmission may be delayed, the influence of transmission delay is considered in the proposed stochastic QSDC scheme. Based on the above, a more general weak infinitesimal operator about the three semi-Markov processes (SMPs) is first given to deal with stochastic Lyapunov functionals (LFs). To reduce the conservatism of the obtained results, stochastic LFs about the two-sided closed-loop functions are constructed in the framework of the sampling patterns ϑ(tk) to ϑ(t) and ϑ(t) to ϑ(tk+1). Then, based on the novel constructed LFs, stochastic stability criterion for SMJMNNs is established by combining with stochastic QSDC law. Finally, the numerical simulation examples are provided to verify the validity and less conservatism of the obtained theoretical results.

Sliding Mode Control for Networked Interval Type-2 Fuzzy Systems via Random Multiaccess Protocols

In this article, the sliding mode control problem is considered for interval type-2 fuzzy systems under the access-constrained communication network. The sensors and actuators are selected in a random manner, under which only a part of sensor/actuator nodes can be permitted to access the communication channels. The access status of the sensor and actuator is described by two independent Markov chains. To deal with the complexity of two Markov chains on the control design, the mapping technique is used to generate a new variable obeying a Markov chain, which can simultaneously reflect the access status of the sensor and actuator. Furthermore, a scheduling signal-dependent fuzzy sliding mode controller is designed, and the MF-dependent sufficient conditions are given to ensure the stochastic stability of the controlled system and the reachability of the sliding surface. Besides, an optimization-based solving algorithm is proposed to search the sliding matrix and obtain the optimized control gains for reducing the energy consumption of control input. Finally, the simulation results of a numerical example and the 2-degree-of-freedom helicopter system are given.

A novel Bayesian method for variable selection and estimation in binary quantile regression

AbstractIn this paper, we develop a Bayesian hierarchical model and associated computation strategy for simultaneously conducting parameter estimation and variable selection in binary quantile regression. We specify customary asymmetric Laplace distribution on the error term and assign quantile‐dependent priors on the regression coefficients and a binary vector to identify the model configuration. Thanks to the normal‐exponential mixture representation of the asymmetric Laplace distribution, we proceed to develop a novel three‐stage computational scheme starting with an expectation–maximization algorithm and then the Gibbs sampler followed by an importance re‐weighting step to draw nearly independent Markov chain Monte Carlo samples from the full posterior distributions of the unknown parameters. Simulation studies are conducted to compare the performance of the proposed Bayesian method with that of several existing ones in the literature. Finally, two real‐data applications are provided for illustrative purposes.

Optimal control‐decision strategy for wireless networked control systems with structural variation and packet dropout

AbstractThis article takes into account a class of wireless networked control systems in which the plant structure is subject to random abrupt variations and signals may be lost due to packet dropout over unreliable transmission channels. Firstly, a discrete‐time Markovian jump linear system with two independent Markov chains is established to model such systems where structural variation is governed by one Markov chain while packet dropout is governed by another. Secondly, a control‐decision strategy is introduced where decision is utilized to decrease signal losses and controller is to guarantee system stability. Meanwhile, a generalized quadratic performance with both control and decision cost is formulated to evaluate the proposed strategy. For the optimization of such generalized performance, an optimal control‐decision strategy, which is dependent on the decision, can then be deduced via stochastic dynamic programming. Based on this, a gradient descent algorithm is proposed for obtaining the optimal decision together with its convergence proof. Numerical examples illustrate the effectiveness of the proposed strategy.

Dominating occupancy processes by the independent site approximation

Occupancy processes are a broad class of discrete time Markov chains on {0,1}n encompassing models from ecology and epidemiology. This model is compared to a collection of n independent Markov chains on {0,1}, which we call the independent site model. We establish conditions under which an occupancy process is smaller in the lower orthant order than the independent site model. An analogous result for spin systems follows by a limiting argument.

A Locally Stationary Markov Chain Model for Labor Dynamics

Labor market surveys usually partition individuals into three states: employed, unemployed, and out of the labor force. In particular, the Argentine “ Encuesta Permanente de Hogares (EPH)” follows a rotating scheme so that each selected household is interviewed four times within two years. Each time, the current labor state of individuals is recorded, together with extensive demographic information. We model those labor paths as consecutive observations from independent Markov chains, were transition matrixes are related to covariates through a multivariate logistic link. Because the EPH is severely affected by attrition, a significant fraction of the surveyed paths contain just one single point. Instead of discarding those observations, we opt to base estimation on the full data by (i) assuming the Markov chains are stationary and (ii) incorporating the chronological time of the first interview as an additional covariate for each individual. This novel treatment represents a convenient approximation, which we illustrate with data from Argentina in the period 1995-2002 via maximum likelihood estimation. Several interesting labor market indexes, which are functionally related to the transition matrixes, are also presented in the last portion of the paper and illustrated with real data.

Fast approximate simulation of finite long-range spin systems

Tau leaping is a popular method for performing fast approximate simulation of certain continuous time Markov chain models typically found in chemistry and biochemistry. This method is known to perform well when the transition rates satisfy some form of scaling behaviour. In a similar spirit to tau leaping, we propose a new method for approximate simulation of spin systems which approximates the evolution of spin at each site between sampling epochs as an independent two-state Markov chain. When combined with fast summation methods, our method offers considerable improvement in speed over the standard Doob–Gillespie algorithm. We provide a detailed analysis of the error incurred for both the number of sites incorrectly labelled and for linear functions of the state.

A double mixture autoregressive model of commodity prices

Many commodity prices exhibit boom-bust type behavior: sustained periods of price increases, followed by sudden sharp collapses. Since around the year 2000, booms have become longer while busts have tended to be short but steep, suggesting a structural change in growth and persistence. We model these features of the data using a novel double mixture autoregression with two independent hidden Markov chains. One chain tracks shifts in mean growth rates that account for rising and falling prices, while a second chain tracks changes in volatility and lag-structure. While the two chains are independent, the persistence of price growth depends on the volatility state, which allows the lag-structure to vary across variance regimes. Estimation requires a two-stage Fisherian approach. Initially, location-related parameters are estimated while suppressing the underlying autoregressive structure. These parameters are then held fixed while the optimal lag-structure across variance regimes is determined. We apply the model to three industrial commodities price time series: Crude Oil, Aluminum, and Rubber. We find that in each case, the model captures boom and bust cycles, with data from more recent periods exhibiting higher volatility, longer price rallies, and steeper collapses.

Hidden Markov Model with Markovian emission

Abstract In our paper [A. Nasroallah and K. Elkimakh, HMM with emission process resulting from a special combination of independent Markovian emissions, Monte Carlo Methods Appl. 23 2017, 4, 287–306] we have studied, in a first scenario, the three fundamental hidden Markov problems assuming that, given the hidden process, the observed one selects emissions from a combination of independent Markov chains evolving at the same time. Here, we propose to conduct the same study with a second scenario assuming that given the hidden process, the emission process selects emissions from a combination of independent Markov chain evolving according to their own clock. Three basic numerical examples are studied to show the proper functioning of the iterative algorithm adapted to the proposed model.

Asynchronous H∞ control for nonhomogeneous higher-level Markov jump systems

This paper investigates about asynchronous H∞ control for nonhomogeneous higher-level Markov jump systems, where a higher-level transition probability matrix is considered. The asynchronous H∞ controller designed is subject to an independent Markov chain. Using a linear matrix inequality approach, sufficient conditions are established such that system with controller is stochastically stable. A numerical example is presented to demonstrate the effectiveness of our method developed.

Two-Mode-Dependent Controller Design for Networked Markov System With Time-Delay in Both S/C Link and C/A Link

The two-mode-dependent controller design problem for networked Markov system with time-delay in both S/C link and C/A link is investigated in this paper. Two independent Markov chains are used to describe the time-delay in S/C link and C/A link. A two-mode-dependent state feedback controller is proposed that depends on both the S/C time-delay and the mode of the Markov controlled plant. The sufficient conditions on the stochastic stability of the closed-loop system are established. The design method of the controller is also proposed on condition that the transition probability matrices of S/C time-delay and mode of the controlled plant are completely known and partly unknown respectively. A numerical example is exploited to illustrate the effectiveness and superiority of the proposed method.

Stabilization for Networked Control System With Time-Delay and Packet Loss in Both S-C Side and C-A Side

The stabilization problem for a class of discrete network control system with time-delay and packet loss in both S-C side and C-A side is researched in this paper. Firstly, two independent discrete Markov chains are used to describe the network time-delay from sensor to controller and the network time-delay from controller to actuator. Two random variables obeying the Bernoulli distribution are employed to describe the packet loss between the sensor and the controller and the packet loss between the controller and the actuator. Secondly, a mathematical model for closed-loop system is established. By constructing the appropriate Lyapunov-Krasovskii functional, the sufficient conditions for the existence of the controller and observer gain matrix are obtained under the condition that the transition probabilities of S-C time-delay and C-A time-delay are both partly unknown. Finally, two examples are exploited to illustrate the effectiveness of the proposed method.

Nonstationary [formula omitted] filtering for Markov switching repeated scalar nonlinear systems with randomly occurring nonlinearities

This paper is concerned with nonstationary l2−l∞ filtering for Markov switching repeated scalar nonlinear systems (MSRSNSs) with randomly occurring nonlinearities (RONs), where measurement output is modeled by a mode-dependent random variable that satisfying Bernouli distribution. The new relationship are proposed to depict multiple mutually independent Markov chains between original MMSRSNSs and nonstationary filters. By constructing a proper Lyapnov function, the MSRSNSs is stochastically stable with l2−l∞ performance level is guaranteed. Accordingly, the nonstationary filters are designed, where filters are characterised by a two-layer structure. The paper provides a numerical example verifying the efficacy of established technique.

Sequential Monte Carlo Samplers with Independent Markov Chain Monte Carlo Proposals

Sequential Monte Carlo (SMC) methods for sampling from the posterior of static Bayesian models are flexible, parallelisable and capable of handling complex targets. However, it is common practice to adopt a Markov chain Monte Carlo (MCMC) kernel with a multivariate normal random walk (RW) proposal in the move step, which can be both inefficient and detrimental for exploring challenging posterior distributions. We develop new SMC methods with independent proposals which allow recycling of all candidates generated in the SMC process and are embarrassingly parallelisable. A novel evidence estimator that is easily computed from the output of our independent SMC is proposed. Our independent proposals are constructed via flexible copula-type models calibrated with the population of SMC particles. We demonstrate through several examples that more precise estimates of posterior expectations and the marginal likelihood can be obtained using fewer likelihood evaluations than the more standard RW approach.

Improved Mode-Dependent State-Feedback Stabilization of Discrete-Time Networked Control Systems with Markovian Communication Delays

The modeling issues in dynamical systems in many processes, networked control systems (NCS) are very complex. The models may contain subsystems with different parameters, which arise when using a network in an NCS such as time delay, limited bandwidth, and so on. Controlling these types of multiple time delay system is challenging, due to mathematical complexity. This paper considers the design of a stabilizing state-feedback controller for a networked control system with random communication delays. Sensor-to-controller (S-C) and controller-to-actuator delays are modeled by two independent Markov chains. Network-induced random delays are modeled as a Markov chain, and the resulting closed-loop system is transformed into a Markovian jump linear system (MJLS). The focus is on the design of a controller that fully incorporates the effect of the C-A delay. The resulting closed-loop system is described by a new discrete-time Markovian jump linear system with Markov delays model. Then, by applying a type of stochastic Lyapunov functional, sufficient conditions on the stochastic stabilizability and the existence of controller are derived in terms of coupled linear matrix inequalities (LMIs). The efficacy of the proposed method is shown through illustrative examples. Simulation results demonstrate the applicability and the effectiveness of the obtained theoretical results.