Order Markov Chain Research Articles

Identifying the dependency pattern of daily rainfall of Dhaka station in Bangladesh using Markov chain and logistic regression model

Bangladesh is a subtropical monsoon climate characterized by wide seasonal variations in rainfall, moderately warm temperatures, and high humidity. Rainfall is the main source of irrigation water everywhere in the Bangladesh where the inhabitants derive their income primarily from farming. Stochastic rainfall models were concerned with the occurrence of wet day and depth of rainfall for different regions to model the daily occurrence of rainfall and achieved satisfactory results around the world. In connection to the Markov chain of different order, logistic regression is conducted to visualize the dependence of current rainfall upon the rainfall of previous two-time period. It had been shown that wet day of the previous two time period compared to the dry day of previous two time period influences positively the wet day of current time period, that is the dependency of dry-wet spell for the occurrence of rain in the rainy season from April to September in the study area. Daily data are collected from meteorological department of about 26 years on rainfall of Dhaka station during the period January 1985-August 2011 to conduct the study. The test result shows that the occurrence of rainfall follows a second order Markov chain and logistic regression also tells that dry followed by dry and wet followed by wet is more likely for the rainfall of Dhaka station and also the model could perform adequately for many applications of rainfall data satisfactorily.

A Novel Method for Evolution Analysis based on Image Registration

Image registration is an important technique in image processing, which could be used to compare the similarity between two images. Here, a novel method based on transition probability matrix of oligonucleotide is proposed to infer the evolutionary relatedness of microbial organisms via image registration. Firstly, the oligonucleotide transition probability matrixes of microbial genomes are calculated by applying 1st order Markov Chain Method. Secondly, each transition probability matrix is converted into a color image, and then combined with each other to a joint histogram. Finally, the point set distribution of joint histogram is analyzed, and divergence formula is introduced and used as the similarity metric, which can reflect the evolutionary relatedness of organisms. For the organisms that taxonomic categories covered from kingdom to species, our results suggest that this new method is more accurate and discriminable than the methods based on single gene or oligonucleotide frequency especially for the classification under species. This method must be broadened and developed so that it can be applied to species identification and phylogeny inferring.

Higher Order Markov Structure-Based Logistic Model and Likelihood Inference for Ordinal Data

Azzalini (1994) proposed a first order Markov chain for binary data. Azzalini’s model is extended for ordinal data and introduces a second order model. Further, the test statistics are developed and the power of the test is determined. An application using real data is also presented.

Ultra-Short-Term Ahead Generating Power Forecasting for PV System Based on Markov Chain for Error Series

This article presents a proper approach for the prediction of output power for photovoltaic generation system. Using the results of solar irradiation from the sunny-day-model as the base value, which are accurate when the sky is clear, and correcting the forecasting results with the assistance of Markov Chain in order to make them valid for other kinds of weathers. To be specific, by utilizing Markov Chain, the underlying stochastic effects of clouds coverage can be minimized and thus harvest more accurate results. The method is tested on the photovoltaic generation system of Electrical Engineering School, Wuhan University, P.R.China and rendered a satisfactory precision of forecasting. Finally, further measurements for enhancing accuracy are also discussed in this paper.

A Computer Algorithm for Order Estimation of Markov Chain with an Application

This paper deals with the problem of estimating the order of Markov chains. An algorithm is proposed for this purpose and programmed by MATLABR2009a, such that the maximum order is selected automatically after inputting of observations and the upper limit of the order. A simulation study is carried out to verify the efficiency of the algorithm, then the algorithm is applied to the observations of rainfall in the city of Mosul for the years 1994-2000 .It is clear from this application that the daily rainfall in Mosul city is with order 1.

Polymerization of styrene with tetradentate chelated α-diimine nickel(II) complexes/MAO catalyst systems: Catalytic behavior and microstructure of polystyrene

The recently reported N,N,N,N-tetradentate chelated α-diimine Ni(II) complexes 1– 3 were demonstrated to be efficient precatalysts for the polymerization of styrene in the presence of MAO, producing isotactic-enriched heterotactic polystyrenes (isotactic contents in the range 73–75.5%). The corresponding sequence distributions were determined by NMR spectroscopy, and fitted to six probabilistic models of chain propagation. A better conformity to a Bovey–first order Markov chain propagation model was obtained, indicating a chain-end control mechanism. A comparison of the microstructural features with other polystyrenes obtained by several different types of initiation indicates that the polystyrenes are formed by a coordination insertion polymerization mechanism at the nickel centers.

Bayesian analysis of variable-order, reversible Markov chains

We define a conjugate prior for the reversible Markov chain of order $r$. The prior arises from a partially exchangeable reinforced random walk, in the same way that the Beta distribution arises from the exchangeable Poly\'{a} urn. An extension to variable-order Markov chains is also derived. We show the utility of this prior in testing the order and estimating the parameters of a reversible Markov model.

Mining Markov chain transition matrix from wind speed time series data

Extracting important statistical patterns from wind speed time series at different time scales is of interest to wind energy industry in terms of wind turbine optimal control, wind energy dispatch/scheduling, wind energy project design and assessment, and so on. In this paper, a systematic way is presented to estimate the first order (one step) Markov chain transition matrix from wind speed time series by two steps. Wind speed time series data is used first to generate basic estimators of transition matrices (i.e. first order, second order, third order, etc.) based on counting techniques. Then an evolutionary algorithm (EA), specifically double-objective evolutionary strategy algorithm (ES), is proposed to search for the first order Markov chain transition matrix which can best match these basic estimators after transforming the first order transition matrix into its higher order counterparts. The evolutionary search for the first order transition matrix is guided by a predefined cost function which measures the difference between the basic estimators and the first order transition matrix, and its high order transformations. To deal with the potential high dimensional optimization problem (i.e. large transition matrices), an enhanced offspring generation procedure is proposed to help the ES algorithm converge efficiently and find better Pareto frontiers through generations. The proposed method is illustrated with wind speed time series data collected from individual 1.5 MW wind turbines at different time scales.

Optimal Pooling of Inventories with Substitution: A Literature Review

In many inventory management systems, some kind of substitution flexibility exists, meaning that a substitute or more flexible item can be used (at an additional cost) when the preferred product stocks out. Through the use of substitution flexibility, we can take advantage of the risk pooling effect on the flexible item. Since risk pooling reduces total inventory holding costs, a trade off between inventory holding costs and flexibility costs will determine the optimal inventory control parameters for the different items. In this research paper, we focus on different types of inventory management systems with substitution flexibility, and discuss three methods suggested in the literature (i.e., newsvendor models, simulation and continuous-time Markov chains) in order to optimally exploit substitution flexibility.

Modelling the profitability of credit cards by Markov decision processes

This paper derives a Markov decision process model for the profitability of credit cards, which allows lenders to find an optimal dynamic credit limit policy. The states of the system are based on the borrower’s behavioural score and the decisions are what credit limit to give the borrower each period. In determining which Markov chain best describes the borrower’s performance, second order as well as first order Markov chains are considered and estimation procedures developed that deal with the low default levels that may exist in the data. A case study is given in which the optimal credit limit is derived and the results compared with the actual outcomes.

Comparability Analyses of the SPI and RDI Meteorological Drought Indices in Different Climatic Zones

Comparability analyses are performed to investigate similarities/differences of the standard precipitation index (SPI) and the reconnaissance drought index (RDI), respectively, utilizing precipitation and ratio of precipitation over potential evapotranspiration (ET0). Data are from stations with different climatic conditions in Iran. Drought characteristics of the 3-month, 6-month and annual SPI and RDI time series are developed and Markov chain order dependencies are investigated by the Log-likelihood, AIC and BIC tests. Steady state probabilities and Markov chain characteristics, i.e., expected residence time in different drought classes and time to reach “Near Normal” class are investigated. According to results, both indices exhibit an overall similar behaviour; particularly, they follow the first order Markov chain dependency. However, climatic variability may produce some differences. In several cases, the “Extremely Dry” class has received a more critical value by RDI. Furthermore, the expected residence time of “Near Normal” class and expected time to reach “Near Normal” class are quite different in a number of cases. The results show that the RDI by utilizing the ET0 can be very sensitive to climatic variability. This is rather important, since if the drought analyses are for agricultural applications, utilization of the RDI would seem to serve a better purpose.

Actively Circulating Volume as a Consequence of Stochasticity within Microcirculation

It is well established that in the pathology of the cardio-vascular system (CVS) only a portion of the blood volume (BV) can be in active circulation. This portion of BV is named the actively circulating volume (ACV) and is evaluated from a monotone decrease of dilution curve produced by an intravascular tracer. In given paper is presented Markov chain as a math model of the flow of a tracer throughout CVS. The consideration of CVS as a set of segments with respect to an anatomical structure and assuming the existence for CVS steady-state condition; leads to the Markov chain of the finite order with constant coefficients. The conclusions of the article are 1) there are open and closed microvessels, such that the switching from open to closed and back is a stochastic process, 2) if the switching is slow then the ACV, as the volume of heart chambers and only open for circulation vessels, can be detected.

Selecting a binary Markov model for a precipitation process

This paper uses rth-order categorical Markov chains to model the probability of precipitation. Several stationary and non-stationary high-order Markov models are proposed and compared using BIC. The number of parameters increases exponentially by adding the Markov order. Several classes of high-order Markov models are proposed which their increase of number of parameters are modest. For example models that use the number of precipitation days in a period prior to date, temperature of the previous day and sines/cosines periodic functions (to model the seasonality) are considered. The theory of partial likelihood is used to estimate the parameters. Parsimonious non-stationary first order Markov models with few seasonal terms are found optimal using BIC and temperature does not turn out to be a useful covariate. However BIC seems to underestimate the number of seasonal terms. We have also compared the results with AIC in some cases which tends to pick parsimonious models with more seasonal terms and higher order. We also show that ignoring seasonal terms result in picking higher order Markov chains. Finally we apply the methods to build confidence intervals for the probability of periods with no precipitation or low number of precipitation days in Calgary using historical data from 1980 to 2000.

Markov Chain Order Estimation to the Weather State of Mosul City by Using Backpropagation Network

The operation of order estimation of a Markov chain to represent the observation chain is an important problem in realistic applications. One method of order estimation depending on intelligence technicality is represented by artificial neural networks . in this research we design an artificial neural network which is the backpropagation error (BPE) . This research treated realistic problem in Markov chain order estimation to the weather state of raining months in Nineveh Governorate ( clearly, cloudy, rainy) , and a special algorithm has been prepared for training the network .We suggest another neural network for backpropagation error which is forecasting the weather state , which is designed depending on the past network after modify it.

Convolution-closed models for count time series with applications

There has recently been an upsurge of interest in time series models for count data. Many papers focus on the model with first-order (Markov) dependence and Poisson innovations. Our paper considers practical models that can capture higher-order dependence based on the work of Joe (1996). In this framework we are able to model both equidispersed and overdispersed marginal distributions of data. The latter is approached using generalized Poisson innovations. Central to the models is the use of the property of closure under convolution of certain families of random variables. The models can be thought of as stationary Markov chains of finite order. Parameter estimation is undertaken by maximum likelihood, inference procedures are considered and means of assessing model adequacy employed. Applications to two new data sets are provided.

Application of Markov chain analysis to a fining-upward fluvial succession of the Early Permian Warchha Sandstone, Salt Range, Pakistan

Markov chain analysis is applied to the cyclic properties and degree of ordering of lithofacies in the Early Permian (Artinskian) Warchha Sandstone. The 30 to 155 m-thick Warchha Sandstone is well exposed in the Salt Range, Pakistan and dominantly composed of sandstone, siltstone and claystone succession. Seven lithofacies have been identified on the basis of geometry, gross lithology and sedimentary structures. Lithofacies are cyclically arranged in a fining-upward pattern. A complete cycle starts with pebbly sandstone accomplished by thin layer of basal conglomerate and terminates with claystone. The non-stationary first order Markov chains have been applied statistically on vertical transition of facies using outcrop data of the Warchha Sandstone succession. Later, a chi-square test was applied to test the dependency between any two facies states in facies transitions. The results of this study reveal that the sediments of the Warchha succession were controlled by Makovian mechanism and as a whole represent fluvial succession deposited in a predictable cyclic arrangement of lithofacies.

Markov Chain Order Estimation to the Weather State of Mosul City by Using Backpropagation Network

Markov Chain Order Estimation to the Weather State of Mosul City by Using Backpropagation Network

On the nucleotide distribution in bacterial DNA sequences

AbstractIt is probable that the distributional structure of DNA sequences arises from the accumulation of many successive stochastic events such as nucleotide deletions, insertions, substitutions and elongations [1, 2, 3, 4, 5, 6, 7]. Although the existence of long-range correlations in non-coding portions of DNA sequences is well established [8, 9, 10, 11], first order Markov chains might well capture aspects of their nucleotide distributions [12]. Here we propose a hidden Markov model based on a coupling of an urn process with a Markov chain to approximate the distributional structure of primitive DNA sequences. Then, by supposing that a bacterial DNA sequence can be derived from uniformly distributed mutations of some primitive DNA, we use the model to explain and predict some distributional properties of bacterial DNA sequences. The distributional properties intrinsic to the model were compared to statistical estimates from 1049 bacterial DNA sequences. In particular, the proposed model provides another possible theoretical explanation for Chargaff’s second parity rule for short oligonucleotides [13, 14].

STOCHASTIC GENERATION OF THE OCCURRENCE AND AMOUNT OF DAILY RAINFALL

Rainfall is the main source of irrigation water in the northwest part of Bangladesh where the inhabitants derive their income primarily from farming. Stochastic rainfall models were concerned with the occurrence of wet day and depth of rainfall. The first order Markov chain model was used to simulate the sequence of rainfall occurrence using the method of transitional probability matrices, while daily rainfall amount was generated using a gamma distribution. The model parameters were estimated from historical rainfall records. The shape and scale parameters were estimated by moment method and hence it became possible to find the parameter values at the study area and then to generate synthetic sequences according to the gamma distribution. The parameters necessary for the whole generation include the means, variance or standard deviation and conditional probabilities of wet and dry days. Results obtained showed that the model could be used to generate rainfall data satisfactorily.

Market behavior and performance of different strategy evaluation schemes

Strategy evaluation schemes are a crucial factor in any agent-based market model, as they determine the agents' strategy preferences and consequently their behavioral pattern. This study investigates how the strategy evaluation schemes adopted by agents affect their performance in conjunction with the market circumstances. We observe the performance of three strategy evaluation schemes, the history-dependent wealth game, the trend-opposing minority game, and the trend-following majority game, in a stock market where the price is exogenously determined. The price is either directly adopted from the real stock market indices or generated with a Markov chain of order ≤2 . Each scheme's success is quantified by average wealth accumulated by the traders equipped with the scheme. The wealth game, as it learns from the history, shows relatively good performance unless the market is highly unpredictable. The majority game is successful in a trendy market dominated by long periods of sustained price increase or decrease. On the other hand, the minority game is suitable for a market with persistent zigzag price patterns. We also discuss the consequence of implementing finite memory in the scoring processes of strategies. Our findings suggest under which market circumstances each evaluation scheme is appropriate for modeling the behavior of real market traders.