Discrete Markov Chain Research Articles

Using Markovian stochastic modelling to predict energy performances and thermal comfort of passive solar systems

A Markovian stochastic approach to the simulation of passive and hybrid solar devices has been developed. The driving variables of the modelled device, as well as the temperatures which characterize its thermal state, are discretized. These quantities are used to set up a random state vector, which defines a discrete homogeneous Markov chain. The transition probabilities of the chain are calculated using the stochastic matrices which are obtained by reduction of weather and users' dependent real sequences of data into the approapriate form. Determination of the long-run distribution of probability of the chain provides for the evaluation of energy performance and thermal comfort indicators. An intermodel comparison, using several internationally well-known deterministic computer programs (DYWON, HELIOS, PASSIM, SERIRES), has been carried out by way of the simulation of a direct-gain office room. This analysis has shown close correspondence between the stochastic and deterministic modelling results. Among others, the main advantage of this approach is the ability of the model to account for the random fluctuations of the driving variables, which affect the thermal performances of the solar device. The development of microcomputer programs based on this approach is regarded as the final goal of this research topic.

About Some Iterative Synchronous and Asynchronous Methods for Markov Chain Distribution Computation

This paper reports on some experiments of parallel processing on a multiprocessor for the numerical computation of the stationary probabilities distribution of the finite state discrete Markov Chain associated to a circuit switched telecommunication network. Several numerical iterative methods are investigated from classical ones to solve linear algebraic systems to more specialized ones to deal with Markov-type systems. The results are discussed with respect to some parameters relevant in parallel processing: number of processors (8 maximum); degree of synchronism, relaxation coefficients,….

Statistical Properties of Generalized Method-of-Moments Estimators of Structural Parameters Obtained From Financial Market Data

The article examines the properties of generalized method of moments GMM estimators of utility function parameters. The research strategy is to apply the GMM procedure to generated data on asset returns from stochastic exchange economies; discrete methods and Markov chain models are used to approximate the solutions to the integral equations for the asset prices. The findings are as follows: (a) There is variance/bias trade-off regarding the number of lags used to form instruments; with short lags, the estimates of utility function parameters are nearly asymptotically optimal, but with longer lags the estimates concentrate around biased values and confidence intervals become misleading, (b) The test of the overidentifying restrictions performs well in small samples; if anything, the test is biased toward acceptance of the null hypothesis.

The Relationship Between a Continuous‐Time Renewal Model and a Discrete Markov Chain Model of Precipitation Occurrence

A relationship between a continuous wet‐dry renewal model with gamma distributed dry intervals and a Markov chain model for daily rainfall is derived. The relationship requires numerical evaluation for the general case, but closed‐form solutions are available for the Markov process case in which dry intervals are exponentially distributed. The Markov process equations are equivalent to those derived by J. R. Green (1964) for first‐order transition probabilities, but are presented in a simplified, nonrecursive form. The Markov process relationships are examined for convergence and accuracy with simulation studies and comparison to observed rainfall records. The Markov process model is shown to provide a good representation in certain parts of the United States. In other areas where the Markov property is inappropriate due to event clustering or other phenomena the gamma model provides an improved characterization of the relationship between continuous and discrete rainfall occurrence.

Likelihood ratio test for testing order of continuous time finite markov chains

Anderson and Goodman ( 1957) have obtained the likelihood ratio tests and chi-square tests for testing the hypothesis about the order of discrete time finite Markov chains, On the similar lines we have obtained likeli¬hood ratio tests and chi-square tests (asymptotic) for testing hypotheses about the order of continuous time Markov chains (MC) with finite state space.

On the principle of minimal correlational entropy

The Onsager-Machlup approximation for a discrete Markov chain, representing multistationary state transitions, is formulated in the limit of small thermal fluctuations. The principle of minimal correlational entropy is used to determine the state in which the invariant probability measure tends to concentrate as the intensity of the fluctuations tends to zero. It is this principle, rather than that of maximum entropy, which is valid in open systems without microscopic reversibility. In systems characterised by microscopic reversibility, the two principles give the same predictions. The principle of minimal correlational entropy is shown to be the statistical analogue of the thermodynamic principle of least dissipation of energy. The principle of minimal correlational entropy is applied to the problem of stochastic exit from domains enclosing multistationary states; it reduces to a minimum entropy difference principle for systems satisfying microscopic reversibility.

Analysis of a multi-microprocessor system with time-shared bus

The method of the multimicroprocessor system analysis is described in this paper. The analysed multimicroprocessor system can be treated as a singular microprocessor with the increased computing power. As a mathematical model the discrete Markov chain is used. This model allows to obtain the required system's parameters such as conflict probability and efficiency. The computing results for multimicroprocessor systems are also presented.

On comparing survival probabilities from discrete observations under unequal censoring

In this paper C(α) tests are derived for the discrete Markov chain set-up that are locally asymptotically most powerful (LAMP) against alternatives with constant odds ratios for the comparison of survival rates under possibly unequal censoring.

Termination of Probabilistic Concurrent Program

The asynchronous execution behavior of several concurrent processes, which may use randomization, is studied. Viewing each process as a discrete Markov chain over the set of common execution states, necessary and sufficient conditions are given for the processes to converge almost surely to a given set of goal states under any fair, but otherwise arbitrary, schedule, provided that the state space is finite. (These conditions can be checked mechanically.) An interesting feature of the proof method is that it depends only on the topology of the transitions and not on the actual values of the probabilities. It is also shown that in this model synchronization protocols that use randomization are in certain cases no more powerful than deterministic protocols. This is demonstrated by (1) establishing lower bounds, similar to those known for deterministic protocols, on the size of a shared variable necessary to ensure mutual exclusion and lockout-free behavior of a randomized protocol and (2) showing that no fully symmetric randomized protocol can ensure mutual exclusion and freedom from lockout. 12 references.

A discrete markov chain representation of The sequential probability ratio test

Properties of the Sequential Probability Ratio Test (SPRT) are approximated by tests which can be represented exactly by discrete Markov chains with absorbing barriers. The approximating test S(d) is constructed by rounding the values of the cumulative sum test statistic of the SPRT to the nearest unit of size d > 0. Exact expressions are given for the properties of S(d) and approximations are obtained using the Perron Frobenius Theorem. The approach can be applied to both truncated and untruncated SPRT's based on either discrete or continuous test statistics. Examples are given for the sequential two sample grouped rank test of Wilcoxon, Rhodes, and Bradley (1963).

Finding the Cyclic Index of an Irreducible, Nonnegative Matrix

The cyclic index $\delta $ of an irreducible nonnegative square matrix is the number of eigenvalues of maximum modulus of that matrix. If $\delta = 1$, the matrix is said to be primitive. The notions of primitivity and cyclic index play an important role in the theory of nonnegative matrices. In the context of discrete Markov chains, the words “period” and “aperiodic” are sometimes used in place of “cyclic index” and “primitive”, respectively. It is known how to test an irreducible nonnegative square matrix for primitivity, but there is no known practical method for finding the cyclic index $\delta $ in the general case. This paper presents a time-optimal algorithm for finding $\delta $.

Upcrossings of discrete hydrologic processes and Markov chains

This paper is concerned with upcrossings of a stochastic process at a specified level. We consider discrete parameter processes whose dependence from the past is limited to a finite number of preceding values, and show how particular Markov chains may be derived from them. In this way upcrossing problems may be studied by means of well-known methods developed for Markov chains. Estimation of involved parameters is discussed, together with identification of the order of dependence from the past. Then forecasting problems are briefly examined, and finally various applications to classic time series and hydrologic data are presented.

Interference Analysis of Shuffle/Exchange Networks

The use of shuffle/exchange (S/E) interconnection networks in multiprocessor systems has been proposed for several applications. In order to evaluate the potential performance and reliability of such systems, the effects of conflicts involving switch and memory contention should be determined. This paper presents a discrete Markov chain model to study the effects of such contention for S/E networks used in random access application environments. This model is used to derive memory bandwidth (MBW) of the system, with analytic expressions presented for 4 X 4 and 8 X 8 S/E networks. Two cases are considered. First, the model is applied to systems in which all processors issue their access requests synchronously, then this model is used to construct a traffic model which allows the generation of access requests at arbitrary times. The results of the analysis are compared with simulation results and the analytical results derived by Bhandarkar for the full Crossbar network. Besides the MBW, other interference-related parameters such as blocking probability, traffic, loading, and blocking delay are discussed. In particular, the loading effect of a 16 X 16 S/E network obtained from the model is compared with the results of the simulation from Wirsching's CONET model.

The theoretical seismic reflection response of sedimentary sequences

A sedimentary sequence can quite often be considered very fruitfully as a discrete Markov chain, and thus can be described by a stochastic matrix. Knowing this matrix and the thickness probability distributions of the layers of the various participating lithologies, while assuming that their densities and velocities are normally distributed, an expression is derived for the autocorrelation (or power spectrum) of the seismic reflection response. This expression can bring out essential characteristics of the power spectra, i.e., the low‐frequency droop and its slope. This then leads to the possible estimation of some physical and sedimentary parameters of the sequence such as the scatter of the reflection coefficients and the sand‐shale ratio or sedimentary cycle thickness. The case of a binary sequence, as the predominant sequence of the sediments of the Gulf Coast area and Nigeria, is analyzed in detail. The theoretical results conform well with field data and lead to the interesting conclusion that indeed the estimation of the above mentioned parameters is possible.

On excess life in certain renewal processes

Excess life distributions for discrete renewal processes may be computed by using elementary discrete Markov chain concepts involving absorption probabilities. Excess life distributions in general may then be obtained by approximating the renewal process under study by a suitably chosen sequence of discrete renewal processes. The technique is illustrated in the cases of renewal processes with interarrival distributions which are a linear combination of two exponentials and uniform [0, 1]. A related algorithm is described for computer generated approximations of excess life distributions corresponding to continuous interarrival time distributions with an increasing c.d.f. Conditions for convergence of this algorithm are examined.

On excess life in certain renewal processes

Excess life distributions for discrete renewal processes may be computed by using elementary discrete Markov chain concepts involving absorption probabilities. Excess life distributions in general may then be obtained by approximating the renewal process under study by a suitably chosen sequence of discrete renewal processes. The technique is illustrated in the cases of renewal processes with interarrival distributions which are a linear combination of two exponentials and uniform [0, 1]. A related algorithm is described for computer generated approximations of excess life distributions corresponding to continuous interarrival time distributions with an increasing c.d.f. Conditions for convergence of this algorithm are examined.

An autoregressive model for multilag Markov chains

By assembling the transition matrix of a finite discrete Markov chain from overlays of matrices which are defined only by the serial correlation coefficients and marginal distribution of the chain to be modelled, a considerable saving is made in the number of parameters required to define a multilag Markov chain. This parsimony is achieved without detriment to the marginal distribution or serial correlation structure of the modelled chain. Applications to daily precipitation sequences and reservoir reliability are outlined to demonstrate the model's versatility.

An autoregressive model for multilag Markov chains

By assembling the transition matrix of a finite discrete Markov chain from overlays of matrices which are defined only by the serial correlation coefficients and marginal distribution of the chain to be modelled, a considerable saving is made in the number of parameters required to define a multilag Markov chain. This parsimony is achieved without detriment to the marginal distribution or serial correlation structure of the modelled chain. Applications to daily precipitation sequences and reservoir reliability are outlined to demonstrate the model's versatility.

The Berry-Esseen theorem for functionals of discrete Markov chains

The error bound O(1/√n) is derived in the central limit theorem for partial sums $$\sum\limits_{j = 1}^n {f(\xi _j )} $$ where ξj is a recurrent discrete Markov chain and f is a real valued function on the state space. In particular it is shown that for bounded f and starting distribution dominated by some multiple of the stationary one, it is sufficient for the chain to have recurrence times with third moments on order to get this bound.

A Note on Finite Homogeneous Continuous-Time Markov Chains

A simple and intuitive method of deriving some properties offinite homogeneous continuoustime Markov chains is given. These chains have absorbing barriers, and arise in the study of population genetics. The method involves the representation of such a process in terms of a discrete Markov chain, and a series of waiting times, which reconstruct the original time scale. An example from population genetics is given.