Random Transition Probabilities Research Articles

Population heterogeneity in the fractional master equation, ensemble self-reinforcement, and strong memory effects.

We formulate a fractional master equationin continuous time with random transition probabilities across the population of random walkers such that the effective underlying random walk exhibits ensemble self-reinforcement. The population heterogeneity generates a random walk with conditional transition probabilities that increase with the number of steps taken previously (self-reinforcement). Through this, we establish the connection between random walks with a heterogeneous ensemble and those with strong memory where the transition probability depends on the entire history of steps. We find the ensemble-averaged solution of the fractional master equationthrough subordination involving the fractional Poisson process counting the number of steps at a given time and the underlying discrete random walk with self-reinforcement. We also find the exact solution for the variance which exhibits superdiffusion even as the fractional exponent tends to 1.

Population heterogeneity in the fractional master equation, ensemble self-reinforcement, and strong memory effects.

We formulate a fractional master equation in continuous time with random transition probabilities across the population of random walkers such that the effective underlying random walk exhibits ensemble self-reinforcement. The population heterogeneity generates a random walk with conditional transition probabilities that increase with the number of steps taken previously (self-reinforcement). Through this, we establish the connection between random walks with a heterogeneous ensemble and those with strong memory where the transition probability depends on the entire history of steps. We find the ensemble-averaged solution of the fractional master equation through subordination involving the fractional Poisson process counting the number of steps at a given time and the underlying discrete random walk with self-reinforcement. We also find the exact solution for the variance which exhibits superdiffusion even as the fractional exponent tends to 1.

Intelligent State Estimation for Continuous Fermenters Using Variational Bayesian Learning

Despite rapid sensor technology developments, monitoring a biological process using regular sensor measurements is challenging, making the process very difficult to characterize. Designing an optimal estimator is an attractive alternative to soft-sensing for such complicated hybrid systems. In this article, the variational Bayesian learning algorithms are proposed to estimate the continuous fermenters' actual states. Special attention is given to the random transition probability matrix (TPM), which is a prerequisite to improving estimation performance. Under the assumption of a time-invariant but random TPM, the Dirichlet distribution is utilized to specify the property of TPM. We then estimate it together with the system state and modal state to approximate the conditional posterior joint distribution. Testing the proposed algorithms using the fermenter model shows that the variational Bayesian learning algorithm can satisfactorily estimate conditions and track TPM in high accuracy.

Multimodal dynamics of nonhomogeneous absorbing Markov chains evolving at stochastic transition rates

The Tokuyama–Mori projection operator method for a reduced time-convolutionless description of a local temporal behavior of an open quantum system interacting with the weakly dissipative and fluctuating pervasive environment is applied to a Markov chain subject to random transition probabilities. The solution to the problem of the multimodal dynamics of a two-stage absorbing Markov chain with the fluctuating forward rate constant augmented by a symmetric dichotomous stochastic process is found exactly and compared with that of the problem for the same Markov chain with the fluctuating backward rate constant. It is shown that these two different tetramodal solutions cannot generally be reduced to but be complementary to each other. In the limit of very frequent fluctuations in forward/backward rate constants of a two-stage absorbing Markov chain, as well as in the case of a one-stage recurrent Markov chain, both solutions become bimodal and superimposed to one another. However, there is a distinction between using of those solutions for the dynamics of a two-stage absorbing Markov chain in the limit of very rare fluctuations at the critical point, in which the former solution shows the resonance effect exhibiting itself as the stochastic immobilization in an initial state, while the latter demonstrates the deterministic decay to the other state.

Reliability modeling for a discrete time multi-state system with random and dependent transition probabilities

In a random environment, state transition probabilities of a multi-state system can change as the environment changes. Thus, a dynamic reliability model with random and dependent transition probabilities is developed for non-repairable discrete-time multi-state system in this article. The dependence among the random state transition probabilities of the system is modeled by a copula function. By probability argument and random process theory, we obtain explicit expressions of some reliability characteristics and joint survival function of random time spent by the system in all working states (partially and completely working states). A special case is considered when the state transition probabilities are dependent random variables with power distribution, and the dependence structure is modeled by Farlie–Gumbel–Morgenstern copula. Numerical examples are also presented to demonstrate the developed model and perform a comparison for the models with random and fixed transition probabilities.

A closed description of the non-autonomous dynamics for an absorbing Markov chain with three states and random transition probabilities

A closed description of the non-autonomous dynamics for an absorbing Markov chain with three states and random transition probabilities

Time-inhomogeneous random Markov chains

We consider Markov chains with random transition probabilities which, moreover, fluctuate randomly with time. We describe such a system by a product of stochastic matrices, , with the factors Mi drawn independently from an ensemble of random Markov matrices, whose columns are independent Dirichlet random variables. The statistical properties of the columns of , its largest eigenvalue and its spectrum are obtained exactly for N = 2 and numerically investigated for general N. For large t, the columns are Dirichlet-distributed, however the distribution is different from the initial one. As for the spectrum, we find that the eigenvalues converge to zero exponentially fast and investigate the statistics of the largest stability and Lyapunov exponents, which are approximated by Gamma distributions. We also observe a concentration of the spectrum on the real line for large t.

Local time of Lévy random walks: A path integral approach.

The local time of a stochastic process quantifies the amount of time that sample trajectories x(τ) spend in the vicinity of an arbitrary point x. For a generic Hamiltonian, we employ the phase-space path-integral representation of random walk transition probabilities in order to quantify the properties of the local time. For time-independent systems, the resolvent of the Hamiltonian operator proves to be a central tool for this purpose. In particular, we focus on the local times of Lévy random walks (Lévy flights), which correspond to fractional diffusion equations.

Updatable, Accurate, Diverse, and Scalable Recommendations for Interactive Applications

Recommender systems form the backbone of many interactive systems. They incorporate user feedback to personalize the user experience typically via personalized recommendation lists. As users interact with a system, an increasing amount of data about a user’s preferences becomes available, which can be leveraged for improving the systems’ performance. Incorporating these new data into the underlying recommendation model is, however, not always straightforward. Many models used by recommender systems are computationally expensive and, therefore, have to perform offline computations to compile the recommendation lists. For interactive applications, it is desirable to be able to update the computed values as soon as new user interaction data is available: updating recommendations in interactive time using new feedback data leads to better accuracy and increases the attraction of the system to the users. Additionally, there is a growing consensus that accuracy alone is not enough and user satisfaction is also dependent on diverse recommendations. In this work, we tackle this problem of updating personalized recommendation lists for interactive applications in order to provide both accurate and diverse recommendations. To that end, we explore algorithms that exploit random walks as a sampling technique to obtain diverse recommendations without compromising on efficiency and accuracy. Specifically, we present a novel graph vertex ranking recommendation algorithm called RP 3 β that reranks items based on three-hop random walk transition probabilities. We show empirically that RP 3 β provides accurate recommendations with high long-tail item frequency at the top of the recommendation list. We also present approximate versions of RP 3 β and the two most accurate previously published vertex ranking algorithms based on random walk transition probabilities and show that these approximations converge with an increasing number of samples. To obtain interactively updatable recommendations, we additionally show how our algorithm can be extended for online updates at interactive speeds. The underlying random walk sampling technique makes it possible to perform the updates without having to recompute the values for the entire dataset. In an empirical evaluation with three real-world datasets, we show that RP 3 β provides highly accurate and diverse recommendations that can easily be updated with newly gathered information at interactive speeds (≪ 100 ms ).

A kind of asymptotic properties of moving averages for Markov chains in Markovian environments

ABSTRACTConsider a Markov chain with finite alphabets. In this paper, we study the asymptotic properties of moving average, harmonic mean, and strong deviation theorems (limit theorems expressed by inequalities) of moving geometric average of random transition probabilities and the generalized entropy ergodic theorem for Markov chains in single infinite Markovian environments. It is shown that, under some mild conditions, the sequence of the generalized relative entropy density converges almost surely and in . The trick of the proofs is the construction of random variables with a parameter and the application of Borel–Cantelli lemma.

Maintenance management of offshore structures using Markov process model with random transition probabilities

The robustness of an offshore engineering design is highly dependent on the maintenance management, where the latter needs a full knowledge of engineering analysis and predictions. An accurate estimation of offshore structural performance with time-varying effect is a keen technical issue. The traditional Markov chain model used for structural strength predictions suffers from the difficulty that some of the measurements or inspection data are largely different from the predicted damage condition. This paper presents a deterioration prediction method for maintenance planning in offshore engineering using the Markov models. Instead of traditional deterministic approaches, the Markov chain model is refined by expressing the transition probabilities as random variables. Through such development, the proposed model is able to estimate an interval for the deterioration of an offshore structure. An existing offshore structure located in South China Sea is used in this study for the demonstration purpose. The selection of transition periods of the Markov chain model is investigated. The use of the stochastic model in the prediction of maintenance timing is also discussed. The results show that the proposed approach can provide more reliable information on structural integrity compared to the conventional method.

MODELING TRAFFIC FLUCTUATIONS AND CONGESTION ON A ROAD NETWORK

Statistical mechanics points out as fluctuations have a relevant role for systems near critical points. We study the effect of traffic fluctuations and the transition to congested states for a stochastic dynamical model of traffic on a road network. The model simulates a finite population that moves from one road to another according to random transition probabilities. In such a way, we mimic the traffic fluctuations due to the granular feature of traffic and the dynamics at the crossing points. Then the amplitude of traffic flow fluctuations is proportional to the average flow as suggested by empirical observations. Assuming a parabolic shaped flow-density relation, there exists an unstable critical point for the road dynamics and the system can perform a phase transition to a congested state, where some roads reach their maximal capacity. We apply a statistical physics approach to study the onset congestion and we characterize analytically the relation between the fluctuations amplitude and the appearance of congested nodes. We verify the results by means of numerical simulations on a Manhattan-like road network. Moreover we point out the existence of oscillating regimes, where traffic oscillations back propagate on the road network, whose onset depend sensitively from the traffic fluctuations and that have a strong influence on the hysteresis cycles of the systems when the traffic load is modulated. The comparison between the numerical simulations and the empirical traffic data recorded by an inductive-loop traffic detector system (MTS system) on the county roads of the Emilia Romagna region in Italy is shortly discussed.

Robust Control, Optimization, and Applications to Markovian Jumping Systems

Robust Control, Optimization, and Applications to Markovian Jumping Systems

Some Limit Properties of the Harmonic Mean of Transition Probabilities for Markov Chains in Markovian Environments Indexed by Cayley's Trees

We prove some limit properties of the harmonic mean of a random transition probability for finite Markov chains indexed by a homogeneous tree in a nonhomogeneous Markovian environment with finite state space. In particular, we extend the method to study the tree-indexed processes in deterministic environments to the case of random enviroments.

Assessment and management of risk in subsurface hydrology: A review and perspective

Uncertainty plagues every effort to model subsurface processes and every decision made on the basis of such models. Given this pervasive uncertainty, virtually all practical problems in hydrogeology can be formulated in terms of (ecologic, monetary, health, regulatory, etc.) risk. This review deals with hydrogeologic applications of recent advances in uncertainty quantification, probabilistic risk assessment (PRA), and decision-making under uncertainty. The subjects discussed include probabilistic analyses of exposure pathways, PRAs based on fault tree analyses and other systems-based approaches, PDF (probability density functions) methods for propagating parametric uncertainty through a modeling process, computational tools (e.g., random domain decompositions and transition probability based approaches) for quantification of geologic uncertainty, Bayesian algorithms for quantification of model (structural) uncertainty, and computational methods for decision-making under uncertainty (stochastic optimization and decision theory). The review is concluded with a brief discussion of ways to communicate results of uncertainty quantification and risk assessment.

A Central Limit Theorem for Random Walk in a Random Environment on a Marked Galton-Watson Tree.

Models of random walks in a random environment were introduced at first by Chernoff in 1967 in order to study biological mechanisms. The original model has been intensively studied since then and is now well understood. In parallel, similar models of random processes in a random environment have been studied. In this article we focus on a model of random walk on random marked trees, following a model introduced by R. Lyons and R. Pemantle (1992). Our point of view is a bit different yet, as we consider a very general way of constructing random trees with random transition probabilities on them. We prove an analogue of R. Lyons and R. Pemantle's recurrence criterion in this setting, and we study precisely the asymptotic behavior, under restrictive assumptions. Our last result is a generalization of a result of Y. Peres and O. Zeitouni (2006) concerning biased random walks on Galton-Watson trees.

Some Limit Properties of Random Transition Probability for Second-Order Nonhomogeneous Markov Chains Indexed by a Tree

We study some limit properties of the harmonic mean of random transition probability for a second-order nonhomogeneous Markov chain and a nonhomogeneous Markov chain indexed by a tree. As corollary, we obtain the property of the harmonic mean of random transition probability for a nonhomogeneous Markov chain.

Analysis of Smoking Cessation Patterns Using a Stochastic Mixed-Effects Model With a Latent Cured State

We develop a mixed model to capture the complex stochastic nature of tobacco abuse and dependence. This model describes transition processes among addiction and nonaddiction stages. An important innovation of our model is allowing an unobserved cure state, or permanent quitting, in contrast to transient quitting. This distinction is necessary to model data from situations where censoring prevents unambiguous determination that a person has been “cured.“ Moreover, the processes that describe transient and permanent quitting are likely to be different and have different policy-making implications. For example, when analyzing factors that influence smoking and can be targeted by interventions, it is more important to target those factors that are associated with permanent quitting rather than transient quitting.We apply our methodology to the Alpha-Tocopherol, Beta-Carotene Cancer Prevention (ATBC) study, a large (29,133 participants) longitudinal cohort study. While ATBC was designed as a cancer prevention study, it contains unique information about the smoking status of each participant during every 4-month period of the study. These data are used to model smoking cessation patterns using a discrete-time stochastic mixed-effects model with three states: smoking, transient cessation, and permanent cessation (absorbent state). Random participant-specific transition probabilities among these states are used to account for participant-to-participant heterogeneity. Another important innovation in our article is to design computationally practical methods for dealing with the size of the dataset and complexity of the models. This is achieved using the marginal likelihood obtained by integrating over the Beta distribution of random effects.

Critical points for spread-out self-avoiding walk, percolation and the contact process above the upper critical dimensions

We consider self-avoiding walk and percolation in ℤd, oriented percolation in ℤd×ℤ+, and the contact process in ℤd, with pD(·) being the coupling function whose range is proportional to L. For percolation, for example, each bond is independently occupied with probability pD(y−x). The above models are known to exhibit a phase transition when the parameter p varies around a model-dependent critical point pc. We investigate the value of pc when d>6 for percolation and d>4 for the other models, and L≫1. We prove in a unified way that pc=1+C(D)+O(L−2d), where the universal term 1 is the mean-field critical value, and the model-dependent term C(D)=O(L−d) is written explicitly in terms of the random walk transition probability D. We also use this result to prove that pc=1+cL−d+O(L−d−1), where c is a model-dependent constant. Our proof is based on the lace expansion for each of these models.

Boundaries and Harmonic Functions for Random Walks with Random Transition Probabilities

The usual random walk on a group (homogeneous both in time and in space) is determined by a probability measure on the group. In a random walk with random transition probabilities this single measure is replaced with a stationary sequence of measures, so that the resulting (random) Markov chains are still space homogeneous, but no longer time homogeneous. We study various notions of measure theoretical boundaries associated with this model and establish an analogue of the Poisson formula for (random) bounded harmonic functions. Under natural conditions on transition probabilities we identify these boundaries for several classes of groups with hyperbolic properties and prove the boundary triviality (i.e., the absence of non-constant random bounded harmonic functions) for groups of subexponential growth, in particular, for nilpotent groups.