Transition Probabilities Of Markov Chain Research Articles

Forecasting and optimal probabilistic scheduling of surplus gas systems in iron and steel industry

To make full use of the gas resource, stabilize the pipe network pressure, and obtain higher economic benefits in the iron and steel industry, the surplus gas prediction and scheduling models were proposed. Before applying the forecasting techniques, a support vector classifier was first used to classify the data, and then the filtering was used to create separate trend and volatility sequences. After forecasting, the Markov chain transition probability matrix was introduced to adjust the residual. Simulation results using surplus gas data from an iron and steel enterprise demonstrate that the constructed SVC-HP-ENN-LSSVM-MC prediction model prediction is accurate, and that the classification accuracy is high under different conditions. Based on this, the scheduling model was constructed for surplus gas operating, and it has been used to investigate the comprehensive measures for managing the operational probabilistic risk and optimize the economic benefit at various working conditions and implementations. It has extended the concepts of traditional surplus gas dispatching systems, and provides a method for enterprises to determine optimal schedules.

Non-saturated IEEE 802.11 networks. A hierarchical 3D Markov model

One of the most widely adopted IEEE 802.11 DCF modeling paradigms is the mean-field Markov model approach. It has its foundations on the analytical two-dimensional Markov model presented by Bianchi, which characterizes the basic access mechanism of DCF under a saturated traffic assumption. This basic approach has been refined by a series of follow-up models that take into account important practical issues not considered in the original model, such as non-saturated traffic load and error-prone channels. However, two-dimensional models are either based on bufferless or infinite buffer network assumptions and thus, they are unable to properly model QoS performance metrics related to finite queues. In this paper, a hierarchical three-dimensional Markov model able to fully capture the Quality of Service (QoS) performance and queueing behavior of non-saturated IEEE 802.11 DCF-based networks is proposed. The third dimension in the hierarchical three-dimensional Markov chain is used to model the queue length and the hierarchy is used to efficiently capture, on one hand, the freezing rules of backoff counters when the broadcast channel is sensed busy and, on the other hand, the error-prone channel conditions. The issue of computational complexity is addressed by devising a computationally efficient method to solve for the steady-state transition probabilities of the 3D Markov chain. The solution for the Markov chain is then exploited to obtain analytical expressions for the whole set of QoS performance metrics of the system, allowing the evaluation/prediction of the impact of buffer size, block error probability and backoff freezing rules on all relevant QoS metrics of a DCF-based IEEE 802.11 network. A close match between analytical and simulation results confirms the validity and effectiveness of the proposed approach.

A robust system reliability analysis using partitioning and parallel processing of Markov chain

AbstractThis paper presents a robust reliability analysis method for systems of multimodular redundant (MMR) controllers using the method of partitioning and parallel processing of a Markov chain (PPMC). A Markov chain is formulated to represent the N distinct states of the MMR controllers. Such a Markov chain has N2 directed edges, and each edge corresponds to a transition probability between a pair of start and end states. Because N can be easily increased substantially, the system reliability analysis may require large computational resources, such as the central processing unit usage and memory occupation. By the PPMC, a Markov chain's transition probability matrix can be partitioned and reordered, such that the system reliability can be evaluated through only the diagonal submatrices of the transition probability matrix. In addition, calculations regarding the submatrices are independent of each other and thus can be conducted in parallel to assure the efficiency. The simulation results show that, compared with the sequential method applied to an intact Markov chain, the proposed PPMC can improve the performance and produce allowable accuracy for the reliability analysis on large-scale systems of MMR controllers.

Some Limit Properties of the Harmonic Mean of Transition Probabilities for Markov Chains in Markovian Environments Indexed by Cayley's Trees

We prove some limit properties of the harmonic mean of a random transition probability for finite Markov chains indexed by a homogeneous tree in a nonhomogeneous Markovian environment with finite state space. In particular, we extend the method to study the tree-indexed processes in deterministic environments to the case of random enviroments.

Modelling species abundance in a river by Negative Binomial hidden Markov models

The investigation of species abundance in rivers involves data which are inherently sequential and unlikely to be fully independent. To take these characteristics into account, a Bayesian hierarchical model within the class of hidden Markov models is proposed to map the distribution of freshwater pearl mussels in the River Dee (Scotland). In order to model the overdispersed series of mussel counts, the conditional probability function of each observation, given the hidden state, is assumed to be Negative Binomial. Both the transition probabilities of the hidden Markov chain and the state-dependent means of the observed process depend on covariates obtained from a hydromorphological survey. Bayesian inference, model choice, and covariate selection based on Markov chain Monte Carlo algorithms are presented. The stochastic selection of the explanatory variables which are associated with a reduced chance of finding a local mussel population provides new evidence for the causes of the deterioration of a highly threatened species.

Modelling of bridge elements deterioration for Serbian bridge inventory

In order to improve bridge management practice for public roads in Serbia, a deterioration model for bridge elements was developed using condition data collected over the last 20 years. The distribution of condition data over condition states allows the estimation of reasonably reliable deterioration models even for advanced deterioration. Based on the literature review, discrete-time Markov chains were chosen for this purpose. Estimating the transition probabilities of discrete Markov chains is straightforward when the time between condition records on bridge elements closely matches the chosen time interval for the Markov chain, which is not the case with condition data in Serbia. To overcome this problem, an expectation maximisation (EM) algorithm has been applied to estimate the transition probabilities. It is shown here that the EM algorithm is a sound and robust method, which can yield a reasonable deterioration model even if limited data are available. In addition, the EM is used to determine the overall bridge deterioration using existing agency rules to derive bridge rating from element condition states.

Stabilization for Networked Control Systems with Random Sampling Periods

This paper investigates the stabilization of networked control systems (NCSs) with random delays and random sampling periods. Sampling periods can randomly switch between three cases according to the high, low, and medium types of network load. The sensor-to-controller (S-C) random delays and random sampling periods are modeled as Markov chains. The transition probabilities of Markov chains do not need to be completely known. A state feedback controller is designed via the iterative linear matrix inequality (LMI) approach. It is shown that the designed controller is two-mode dependent and depends on not only the currentS-Cdelay but also the most recent available sampling period at the controller node. The resulting closed-loop systems are special discrete-time jump linear systems with two modes. The sufficient conditions for the stochastic stability are established. An example of the cart and inverted pendulum is given to illustrate the effectiveness of the theoretical result.

Asymptotic analysis of the closed network with time-dependent service parameters and single-type messages

Journal of Applied Mathematics and Computational Mechanics, Prace Naukowe Instytutu Matematyki i Informatyki, Politechnika Częstochowska, Scientific Research of the Institute of Mathematics and Computer Science, Czestochowa University of Technology

Delay-dependent stability analysis for discrete-time singular Markovian jump systems with time-varying delay

This article considers the problem of delay-dependent stability analysis for a class of discrete-time singular systems with Markovian jump and time-varying delay. The transition probabilities in Markov chain are assumed to be partially unknown. In terms of linear matrix inequality approach, the delay-dependent criteria are proposed to ensure the underlying system to be regular, causal and stochastically stable. Several numerical examples are given to demonstrate the effectiveness and less conservatism of the obtained results.

Inference on Partially Observed Quasi-stationary Markov Chains With Applications to Multistate Population Models

If the full capture histories of captured individuals are available, inferences on multistate open population models may be conducted using the well known Arnason–Schwarz model. However, data of this detail is not always available. It is well known that inference on the transition probabilities of a Markov chain may be conducted using aggregate data and we extend this approach to aggregate data on multistate open population models. We show that for parameters to be identifiable we need to augment the aggregate data and we achieve this by batch marking a cohort of individuals according to their initial state, so that the batch marking augments the aggregate data. Model performance is examined by conducting several simulation studies and the model is applied to a real data set where full capture histories are available so it may be compared with the Arnason–Schwarz estimates. This article has supplementary material online.

Logistic Regression Models for Higher Order Transition Probabilities of Markov Chain for Analyzing the Occurrences of Daily Rainfall Data

Logistic regression models for transition probabilities of higher order Markov models are developed for the sequence of chain dependent repeated observations. To identify the significance of these models and their parameters a test procedure for a likelihood ratio criterion is developed. A method of model selection is suggested on the basis of AIC and BIC procedures. The proposed models and test procedures are applied to analyze the occurrences of daily rainfall data for selected stations in Bangladesh. Based on results from these models, the transition probabilities of first order Markov model for temperature and humidity provided the most suitable option to model forecasts for daily rainfall occurrences for five selected stations in Bangladesh.

An Innovative Hybrid Algorithm for Very Short-Term Wind Speed Prediction Using Linear Prediction and Markov Chain Approach

A new hybrid algorithm using linear prediction and Markov chain is proposed in order to facilitate very short-term wind speed prediction. First, the Markov chain transition probability matrix is calculated. Then, linear prediction method is applied to predict very short-term values. Finally, the results are modified according to the long-term pattern by a nonlinear filter. The results from proposed method are compared by linear prediction method, persistent method and actual values. It is shown that the prediction-modification processes improves very short-term predictions, by reducing the maximum percentage error and mean absolute percentage error, while it retains simplicity and low CPU time and improvement in uncertainty of prediction.

The impact of wind power plants on slow voltage variations in distribution networks

The connection of a wind power plant to a distribution network affects the voltage conditions in that part of the network. However, rather than assessing the worst possible cases regarding the voltage range, in this paper a more accurate methodology is presented for calculating the annual distribution network steady-state voltage profile and slow voltage variations after wind power plant connection. Commonly used simulation methods for the same problem based on probabilistic load flow analysis are only able to give probability distributions of the node voltages and network power flows. Here, a method is proposed which can also give chronological insight into the considered network state. This is realized through wind speed data modeling and generation using a Markov chain transition probability matrix and the Monte Carlo simulation method. The proposed methodology is tested using data from an existing distribution network in Croatia, a wind power plant planned for connection, and one year wind data measurements.

An Evolutionary Reduction Principle for Mutation Rates at Multiple Loci

A model of mutation rate evolution for multiple loci under arbitrary selection is analyzed. Results are obtained using techniques from Karlin (Evolutionary Biology, vol. 14, pp. 61-204, 1982) that overcome the weak selection constraints needed for tractability in prior studies of multilocus event models.A multivariate form of the reduction principle is found: reduction results at individual loci combine topologically to produce a surface of mutation rate alterations that are neutral for a new modifier allele. New mutation rates survive if and only if they fall below this surface-a generalization of the hyperplane found by Zhivotovsky et al. (Proc. Natl. Acad. Sci. USA 91, 1079-1083, 1994) for a multilocus recombination modifier. Increases in mutation rates at some loci may evolve if compensated for by decreases at other loci. The strength of selection on the modifier scales in proportion to the number of germline cell divisions, and increases with the number of loci affected. Loci that do not make a difference to marginal fitnesses at equilibrium are not subject to the reduction principle, and under fine tuning of mutation rates would be expected to have higher mutation rates than loci in mutation-selection balance.Other results include the nonexistence of 'viability analogous, Hardy-Weinberg' modifier polymorphisms under multiplicative mutation, and the sufficiency of average transmission rates to encapsulate the effect of modifier polymorphisms on the transmission of loci under selection. A conjecture is offered regarding situations, like recombination in the presence of mutation, that exhibit departures from the reduction principle. Constraints for tractability are: tight linkage of all loci, initial fixation at the modifier locus, and mutation distributions comprising transition probabilities of reversible Markov chains.

Stress testing of retail mortgages: a study based on non-stationary Markov chains and t-copula simulation

In this study, real-life retail mortgage loan data from a Chinese national commercial bank is used to generate the projected distribution over a set of predefined mortgage states or categories. Non-stationary Markov chain transition probabilities between these states are calculated using loan data from 57 consecutive months. In order to validate the model and assess the risk, we used the data gathered to simulate, by a t-copula method, the projected portfolio distribution over the states of a retail mortgage loan in different shock scenarios. The approach proposed in this paper can be readily extended to other retail credit products as well.

Estimation in Reversible Markov Chains

This article examines estimation of the one-step-ahead transition probabilities in a reversible Markov chain on a countable state space. A symmetrized moment estimator is proposed that exploits the reversible structure. Examples are given where the symmetrized estimator has superior asymptotic properties to those of a naive estimator, implying that knowledge of reversibility can sometimes improve estimation. The asymptotic mean and variance of the estimators are quantified. The results are proven using only elementary results such as the law of large numbers and the central limit theorem.

Maximum Entropy Estimation of Transition Probabilities of Reversible Markov Chains

In this paper, we develop a general theory for the estimation of the transition probabilities of reversible Markov chains using the maximum entropy principle. A broad range of physical models can be studied within this approach. We use one-dimensional classical spin systems to illustrate the theoretical ideas. The examples studied in this paper are: the Ising model, the Potts model and the Blume-Emery-Griffiths model.

Markov chain models to estimate the premium for extended hedge fund lockups

AbstractA lockup period for investment in a hedge fund is a time period after making the investment during which the investor cannot freely redeem his investment. It is routine to have a one‐year lockup period, but recently the requested lockup periods have grown longer. We estimate the premium for such extended lockup, taking the point of view of a manager of a fund of funds, who has to choose between two investments in similar funds in the same strategy category, the first having a one‐year lockup and the second having an n‐year lockup. Assuming that the manager will rebalance his portfolio of hedge funds on a yearly basis, if permitted, we define the annual lockup premium as the difference between the rates of return from these investments. We develop a Markov chain model to estimate this lockup premium. By solving systems of equations, we fit the Markov chain transition probabilities to three directly observable hedge fund performance measures: the persistence of return, the variance of return and the hedge‐fund death rate. The model quantifies the way the lockup premium depends on these parameters. Data from the TASS database are used to estimate the persistence, which is found to be statistically significant. Copyright © 2009 Wilmott Magazine Ltd

IMPRECISE MARKOV CHAINS AND THEIR LIMIT BEHAVIOR

When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This can be done by considering as basic uncertainty models the so-calledcredal setsthat these probabilities are known or believed to belong to and by allowing the probabilities to vary over such sets. This leads to the definition of animprecise Markov chain. We show that the time evolution of such a system can be studied very efficiently using so-calledlowerandupper expectations, which are equivalent mathematical representations of credal sets. We also study how the inferred credal set about the state at timenevolves asn→∞: under quite unrestrictive conditions, it converges to a uniquely invariant credal set, regardless of the credal set given for the initial state. This leads to a non-trivial generalization of the classical Perron–Frobenius theorem to imprecise Markov chains.

Randomized Shortest-Path Problems: Two Related Models

This letter addresses the problem of designing the transition probabilities of a finite Markov chain (the policy) in order to minimize the expected cost for reaching a destination node from a source node while maintaining a fixed level of entropy spread throughout the network (the exploration). It is motivated by the following scenario. Suppose you have to route agents through a network in some optimal way, for instance, by minimizing the total travel cost-nothing particular up to now-you could use a standard shortest-path algorithm. Suppose, however, that you want to avoid pure deterministic routing policies in order, for instance, to allow some continual exploration of the network, avoid congestion, or avoid complete predictability of your routing strategy. In other words, you want to introduce some randomness or unpredictability in the routing policy (i.e., the routing policy is randomized). This problem, which will be called the randomized shortest-path problem (RSP), is investigated in this work. The global level of randomness of the routing policy is quantified by the expected Shannon entropy spread throughout the network and is provided a priori by the designer. Then, necessary conditions to compute the optimal randomized policy-minimizing the expected routing cost-are derived. Iterating these necessary conditions, reminiscent of Bellman's value iteration equations, allows computing an optimal policy, that is, a set of transition probabilities in each node. Interestingly and surprisingly enough, this first model, while formulated in a totally different framework, is equivalent to Akamatsu's model ( 1996 ), appearing in transportation science, for a special choice of the entropy constraint. We therefore revisit Akamatsu's model by recasting it into a sum-over-paths statistical physics formalism allowing easy derivation of all the quantities of interest in an elegant, unified way. For instance, it is shown that the unique optimal policy can be obtained by solving a simple linear system of equations. This second model is therefore more convincing because of its computational efficiency and soundness. Finally, simulation results obtained on simple, illustrative examples show that the models behave as expected.