Order Markov Chain Research Articles

Planets and Stellar Activity: Hide and Seek in the CoRoT-7 system

AbstractSince the discovery of the transiting Super-Earth CoRoT-7b, several investigations have been made of the number and precise masses of planets present in the system, but they all yield different results, owing to the star's high level of activity. Radial velocity (RV) variations induced by stellar activity therefore need to be modelled and removed to allow a reliable detection of all planets in the system. We re-observed CoRoT-7 in January 2012 with both HARPS and the CoRoT satellite, so that we now have the benefit of simultaneous RV and photometric data. We fitted the off-transit variations in the CoRoT lightcurve using a harmonic decomposition similar to that implemented in Queloz et al. (2009). This fit was then used to model the stellar RV contribution, according to the methods described by Aigrain et al. (2011). This model was incorporated into a Monte Carlo Markov Chain in order to make a precise determination of the orbits of CoRoT-7b and CoRoT-7c. We also assess the evidence for the presence of one or two additional planetary companions.

Delayed feedback makes neuronal firing statistics non-Markovian

The instantaneous state of a neural network consists of both the degree of excitation of each neuron and the positions of impulses in communication lines between the neurons. In neurophysiological experiments, the times of neuronal firing are recorded but not the state of communication lines. However, future spiking moments substantially depend on the past positions of impulses in the lines. This suggests that the sequence of intervals between firing moments (interspike intervals, ISI) in the network can be non-Markovian. In the present paper, we analyze this problem for the simplest possible neural “network,” namely, for a single neuron with delayed feedback. The neuron receives excitatory input both from the input Poisson process and from its own output through the feedback line. We obtain exact expressions for the conditional probability density P(t n+1 | t n ,…,t 1, t 0)dt n+1 and prove that P(t n+1 | t n ,…,t 1, t 0) is not reduced to P(t n+1 | t n ,…,t 1) for any n ≥ 0: This means that the output ISI stream cannot be represented as a Markov chain of any finite order.

Spatial and temporal scales of new particle formation events in eastern North America

New particle formation (NPF) events have been observed in numerous locations. However, questions remain as to the scale of the events and their importance to regional and global particle number concentrations, size distributions and climate forcing. This study presents measured particle size distributions (PSD) at multiple sites across eastern North America and evaluates the degree of coherence on large (hundreds of kilometer) scales and the site-to-site variability across scales of tens of kilometers. Long-term data from sites separated by 1500 km demonstrate frequent and synchronous NPF, that over 80% of event days at both sites are followed by another event day and that event sequences are best described by a Markov Chain of order 1. Estimates of the mean spatial scale of NPF from a site in southern Indiana range from at least 120–850 km depending on the season and the precise assumptions applied. Despite the evidence for regional coherence in NPF, detailed measurements along an 80 km transect in southern Indiana also indicate some important sub-regional variability. While PSD from individual days typically indicate NPF at all three sites or at none of the sites, PSD measured in two urban environments show greater coherence than those from a centrally located site in a forest, and both the number of ultrafine particles and their growth rates are typically (but not uniformly) higher at the forested site. Some of the site-to-site variability appears to be causally linked to planetary boundary layer dynamics and variations in land cover.

On the limiting probability distribution of a transition probability tensor

In this paper, we propose and develop an iterative method to calculate a limiting probability distribution vector of a transition probability tensor arising from a higher order Markov chain. In the model, the computation of such limiting probability distribution vector can be formulated as a -eigenvalue problem associated with the eigenvalue 1 of where all the entries of are required to be non-negative and its summation must be equal to one. This is an analog of the matrix case for a limiting probability vector of a transition probability matrix arising from the first-order Markov chain. We show that if is a transition probability tensor, then solutions of this -eigenvalue problem exist. When is irreducible, all the entries of solutions are positie. With some suitable conditions of , the limiting probability distribution vector is even unique. Under the same uniqueness assumption, the linear convergence of the iterative method can be established. Numerical examples are presented to illustrate the theoretical results of the proposed model and the iterative method.

Firing statistics of inhibitory neuron with delayed feedback. II: Non-Markovian behavior

The instantaneous state of a neural network consists of both the degree of excitation of each neuron the network is composed of and positions of impulses in communication lines between the neurons. In neurophysiological experiments, the neuronal firing moments are registered, but not the state of communication lines. But future spiking moments depend essentially on the past positions of impulses in the lines. This suggests, that the sequence of intervals between firing moments (inter-spike intervals, ISIs) in the network could be non-Markovian.In this paper, we address this question for a simplest possible neural “net”, namely, a single inhibitory neuron with delayed feedback. The neuron receives excitatory input from the driving Poisson stream and inhibitory impulses from its own output through the feedback line. We obtain analytic expressions for conditional probability density P(tn+1|tn, …, t1, t0), which gives the probability to get an output ISI of duration tn+1 provided the previous (n+1) output ISIs had durations tn, …, t1, t0. It is proven exactly, that P(tn+1|tn, …, t1, t0) does not reduce to P(tn+1|tn, …, t1) for any n≥0. This means that the output ISIs stream cannot be represented as a Markov chain of any finite order.

Exact Goodness‐of‐Fit Tests for Markov Chains

Goodness-of-fit tests are useful in assessing whether a statistical model is consistent with available data. However, the usual χ² asymptotics often fail, either because of the paucity of the data or because a nonstandard test statistic is of interest. In this article, we describe exact goodness-of-fit tests for first- and higher order Markov chains, with particular attention given to time-reversible ones. The tests are obtained by conditioning on the sufficient statistics for the transition probabilities and are implemented by simple Monte Carlo sampling or by Markov chain Monte Carlo. They apply both to single and to multiple sequences and allow a free choice of test statistic. Three examples are given. The first concerns multiple sequences of dry and wet January days for the years 1948-1983 at Snoqualmie Falls, Washington State, and suggests that standard analysis may be misleading. The second one is for a four-state DNA sequence and lends support to the original conclusion that a second-order Markov chain provides an adequate fit to the data. The last one is six-state atomistic data arising in molecular conformational dynamics simulation of solvated alanine dipeptide and points to strong evidence against a first-order reversible Markov chain at 6 picosecond time steps.

On the spectral analysis of second-order Markov chains

Second order Markov chains which are trajectorially reversible are considered. Contrary to the reversibility notion for usual Markov chains, no symmetry property can be deduced for the corresponding transition operators. Nevertheless and even if they are not diagonalizable in general, we study some features of their spectral decompositions and in particular the behavior of the spectral gap under appropriate perturbations is investigated. Our quantitative and qualitative results confirm that the resort to second order Markov chains is an interesting option to improve the speed of convergence to equilibrium.

Modeling and performance of movement-based registration in wireless network

Various location registration schemes have been proposed to minimize the signaling cost on radio channels. In this study, we consider movement-based registration (MBR). MBR is simple and its implementation is quite straightforward. However, it may exhibit relatively frequent location registrations compared to other methods. To solve this problem, a new MBR scheme combined with implicit registration (MBIR) was proposed and analyzed by using the continuous-time Markov chain (CTMC) model. However, previously developed CTMC models have some inherent flaws. Therefore, we propose a new model using an imbedded Markov chain (IMMC) in order to analyze the exact performance of movement-based registration schemes. The numerical results obtained with the proposed IMMC model are compared with those from previous CTMC models.

Rolling Generation Dispatch Based on Ultra-short-term Wind Power Forecast

The power systems economic and safety operation considering large-scale wind power penetration are now facing great challenges, which are based on reliable power supply and predictable load demands in the past. A rolling generation dispatch model based on ultra-short-term wind power forecast was proposed. In generation dispatch process, the model rolling correct not only the conventional units power output but also the power from wind farm, simultaneously. Second order Markov chain model was utilized to modify wind power prediction error state (WPPES) and update forecast results of wind power over the remaining dispatch periods. The prime-dual affine scaling interior point method was used to solve the proposed model that taken into account the constraints of multi-periods power balance, unit output adjustment, up spinning reserve and down spinning reserve.

Markov chain order estimation with conditional mutual information

We introduce the Conditional Mutual Information (CMI) for the estimation of the Markov chain order. For a Markov chain of K symbols, we define CMI of order m, Ic(m), as the mutual information of two variables in the chain being m time steps apart, conditioning on the intermediate variables of the chain. We find approximate analytic significance limits based on the estimation bias of CMI and develop a randomization significance test of Ic(m), where the randomized symbol sequences are formed by random permutation of the components of the original symbol sequence. The significance test is applied for increasing m and the Markov chain order is estimated by the last order for which the null hypothesis is rejected. We present the appropriateness of CMI-testing on Monte Carlo simulations and compare it to the Akaike and Bayesian information criteria, the maximal fluctuation method (Peres–Shields estimator) and a likelihood ratio test for increasing orders using ϕ-divergence. The order criterion of CMI-testing turns out to be superior for orders larger than one, but its effectiveness for large orders depends on data availability. In view of the results from the simulations, we interpret the estimated orders by the CMI-testing and the other criteria on genes and intergenic regions of DNA chains.

Accurate and effective inference of network link loss from unicast end-to-end measurements

Understanding the network link loss is particularly important for optimising delay-sensitive applications. This study addresses the issue of estimating temporal dependence characteristic of link loss by using network tomography. Different from existing works of network loss tomography, the authors use a kth order Markov Chain (k-MC for short, k>1) to model the packet loss process, and propose a constrained optimisation-based method to estimate the state transmission probabilities of the k-MC link loss model. The authors also propose a top–down algorithm in order to ensure that our method can be applied to large networks. Compared with existing loss tomography methods, our method is capable of obtaining more accurate packet loss probability estimates. The ns-2 simulation results show the good performance of our method.

Predicting Drought Magnitudes: A Parsimonious Model for Canadian Hydrological Droughts

A multiplicative relationship, drought magnitude (M) = drought intensity (I) × drought duration or length (L) is used as a basis for predicting the largest expected value of hydrological drought magnitude, E(M T ) over a period of T-year (or month). The prediction of E(M T ) is carried out in terms of the SHI (standardized hydrological index, tantamount to standard normal variate) sequences of the annual and monthly streamflow time series. The probability distribution function (pdf) of I (drought intensity) was assumed to follow a truncated normal. The drought length (L c ) was taken as some characteristic duration of the drought period, which is expressible as a linear combination of the expected longest (extreme) duration, E(L T ) and the mean duration, L m of droughts and is estimated involving a parameter o (range 0 to 1). The drought magnitude (deficit-sum, M) has been assumed to follow a gamma pdf, in view of the observed behavior of M. The model M = I × L has been invoked via two approximations, viz. Type-1 involves only mean of I and Type-2 involves both mean and variance of I through the theorem of extremes of random numbers of random variables. The E(L T ) were obtained using the Markov chain (MC) model of an appropriate order, which turned out to be zero order Markov chain (MC-0) at the annual time scale. At the monthly time scale, the E(L T ) was best represented by MC-0 for SHI sequences with low value of lag-1 autocorrelation (ρ 0.3. At low cutoff levels (q ≤ 0.2), the trivial relationship E(M T ) = E(I) × E(L T ) i.e. without considerations of the extreme number theorem and the pdf of M yielded satisfactory results.

Forecasting Model Based on Improved Grey-Markov

It is great significance to forecast the water quality of Poyang Lake in the construction of Poyang Lake Ecological Economic Zone.On the condition of limited historical data,for resolving the prediction,by combining the advantages of both grey prediction and Markov theory,an improved grey-Markov model is established.First the sequence is moved average,and then unbiased grey-Markov prediction is established to imitate the development tendency of the forecast system.Divide system state into five parts by Markov chain in order to inprove the prediction accuracy. By analying an example,comparing the predicted values and real measured values,The results show that the model has a certain accuracy and reliability

A stationarity test on Markov chain models based on marginal distribution

A stationarity test on Markov chain models is proposed in this paper. Most of the previous test procedures for the Markov chain models have been done based on the conditional probabilities of a transition matrix. The likelihood ratio and Pearson type chi-square tests have been used for testing stationarity and order of Markov chains. This paper uses the efficient score test, an extension of the test developed by Tsiatis (1980) [18], for testing the stationarity of Markov chain models based on the marginal distribution as obtained by Azzalini (1994) [2]. For testing the suitability of the proposed method, a numerical example of real life data and simulation studies for comparison with an alternative test procedure are given.

Comparison of in-situ delay monitors for use in Adaptive Voltage Scaling

Abstract. In Adaptive Voltage Scaling (AVS) the supply voltage of digital circuits is tuned according to the circuit's actual operating condition, which enables dynamic compensation to PVTA variations. By exploiting the excessive safety margins added in state-of-the-art worst-case designs considerable power saving is achieved. In our approach, the operating condition of the circuit is monitored by in-situ delay monitors. This paper presents different designs to implement the in-situ delay monitors capable of detecting late but still non-erroneous transitions, called Pre-Errors. The developed Pre-Error monitors are integrated in a 16 bit multiplier test circuit and the resulting Pre-Error AVS system is modeled by a Markov chain in order to determine the power saving potential of each Pre-Error detection approach.

Reactivity Ratios of Comonomers from a Single MALDI–ToF–MS Measurement at One Feed Composition

The reactivity ratios in a copolymerization are needed to predict the microstructure (random, gradient, block or alternating) of the produced copolymer. This microstructure reflects on the physical properties of the polymeric material. Conventional ways to determine these reactivity ratios demand in most cases tedious laboratory work and several experiments at different monomer feed compositions. Here, a novel method is described to derive these ratios from a single MALDI–ToF–MS spectrum obtained at one feed composition by employing either a Monte Carlo approach to numerically simulate a first order Markov chain or the analytical form of the first order Markov chain. A single MALDI–ToF–MS spectrum proved to give very good estimates of the reactivity ratios of comonomers from copolymer’s synthesized by free radical polymerization, ring-opening polymerization of lactones and lactides, or ring-opening copolymerization of anhydrides plus epoxides.

Malware Detection using Windows API Sequence and Machine Learning

onitoring the behavior of program execution at run-time is widely used to differentiate benign and malicious processes executing in the host computer. Most of the existing run-time malware detection methods use the information available in Windows Application Programming Interface (API) calls. The proposed malware detection system uses the Windows API call sequence. A 3rd order Markov chain (i.e. 4-grams) is used to model the API calls. This composite feature set is provided as an input to the malware detection system to raise the final alarm. Association mining based classification is used because it yields higher detection accuracy than previous data mining based detection systems which employed Naive Bayes, Support Vector Machine and Decision Tree techniques. A minimal subset of API categories is monitored while maintaining high detection accuracy. The number of generated rules is reduced, by removing the redundant rules, to make the malware analysis efficient. The key novelty of the proposed malware detection system is the iterative learning process combined with the run-time monitoring of program execution behavior which makes this as a dynamic malware detection system. The performance of the proposed malware detection system is evaluated for accuracy of malware detection system and compared with the existing data mining based detection systems. It is inferred that the proposed malware detection system outperforms the existing malware detection systems.

New life‐cycle costing approach for infrastructure rehabilitation

PurposeSeveral rehabilitation planning methods are reported in the literature for public infrastructures, such as bridges, pavements, sewers, etc. These methods, however, are limited to specific types of infrastructures. The purpose of the present research is to develop a novel and generic method for Maintenance and Rehabilitation Planning for Public Infrastructure (M&RPPI), which aims at determining the optimal rehabilitation profile over a desired analysis period.Design/methodology/approachThe M&RPPI method is based on life‐cycle costing (LCC) with probabilistic and continuous rating approach for condition states. The M&RPPI uses a new approach of “dynamic” Markov chain to represent the deterioration mechanism of an infrastructure and the impact of rehabilitation interventions on such infrastructure. It also uses genetic algorithm (GA) in conjunction with Markov chains in order to find the optimal rehabilitation profile. A case study is presented with a comparison between the traditional Markov decision process (MDP) and the newly developed method.FindingsThe new method, which generates lower LCC, is found practical in providing a complete M&R plan over a required study period, compared to a stationary decision policy with the traditional MDP. In addition, GA is found useful in the optimization process and overcomes the computational difficulties for large combinatorial problems.Research limitations/implicationsThe implementation of the developed models is limited to only four alternatives/actions. However, the developed models and framework are superior for MDP.Practical implicationsThe developed methodology and model play essential roles in the decision‐making process.Originality/valueThe new method is beneficial to researchers and practitioners. It is developed for a single facility; however, it provides a major step towards a broader infrastructure management system and capital budgeting problems.

Temporal Dependence Network Link Loss Inference from Unicast End-to-End Measurements

In this letter, we address the issue of estimating the temporal dependence characteristic of link loss by using network tomography. We use a k-th order Markov chain (k>1) to model the packet loss process, and estimate the state transition probabilities of the link loss model using a constrained optimization-based method. Analytical and simulation results indicate that our method yields more accurate packet loss probability estimates than existing loss inference methods.

Identifying the dependency pattern of daily rainfall of Dhaka station in Bangladesh using Markov chain and logistic regression model

Bangladesh is a subtropical monsoon climate characterized by wide seasonal variations in rainfall, moderately warm temperatures, and high humidity. Rainfall is the main source of irrigation water everywhere in the Bangladesh where the inhabitants derive their income primarily from farming. Stochastic rainfall models were concerned with the occurrence of wet day and depth of rainfall for different regions to model the daily occurrence of rainfall and achieved satisfactory results around the world. In connection to the Markov chain of different order, logistic regression is conducted to visualize the dependence of current rainfall upon the rainfall of previous two-time period. It had been shown that wet day of the previous two time period compared to the dry day of previous two time period influences positively the wet day of current time period, that is the dependency of dry-wet spell for the occurrence of rain in the rainy season from April to September in the study area. Daily data are collected from meteorological department of about 26 years on rainfall of Dhaka station during the period January 1985-August 2011 to conduct the study. The test result shows that the occurrence of rainfall follows a second order Markov chain and logistic regression also tells that dry followed by dry and wet followed by wet is more likely for the rainfall of Dhaka station and also the model could perform adequately for many applications of rainfall data satisfactorily.