Discrete Markov Chain Research Articles

Survival probabilities in the Sisyphus random walk model with absorbing traps

We analyze the dynamics of the Sisyphus random walk model, a discrete Markov chain in which the walkers may randomly return to their initial position x0. In particular, we present a remarkably compact derivation of the time-dependent survival probability function S(t;x0) which characterizes the random walkers in the presence of an absorbing trap at the origin. The survival probabilities are expressed in a compact mathematical form in terms of the x0-generalized Fibonacci-like numbers Gt(x0). Interestingly, it is proved that, as opposed to the standard random walk model in which the survival probabilities depend linearly on the initial distance x0 of the walkers from the trap and decay asymptotically as an inverse power of the time, in the Sisyphus random walk model the asymptotic survival probabilities decay exponentially in time and are characterized by a non-trivial (non-linear) dependence on the initial gap x0 from the absorbing trap. We use the analytically derived results in order to analyze the underlying dynamics of the ‘survival-game’, a highly risky investment strategy in which non-absorbed agents receive a reward P(t) which gradually increases in time.

Identifying category representations for complex stimuli using discrete Markov chain Monte Carlo with people

With the explosion of “big data,” digital repositories of texts and images are growing rapidly. These datasets present new opportunities for psychological research, but they require new methodologies before researchers can use these datasets to yield insights into human cognition. We present a new method that allows psychological researchers to take advantage of text and image databases: a procedure for measuring human categorical representations over large datasets of items, such as arbitrary words or pictures. We call this method discrete Markov chain Monte Carlo with people (d-MCMCP). We illustrate our method by evaluating the following categories over datasets: emotions as represented by facial images, moral concepts as represented by relevant words, and seasons as represented by images drawn from large online databases. Three experiments demonstrate that d-MCMCP is powerful and flexible enough to work with complex, naturalistic stimuli drawn from large online databases.

A decomposition method for assembly/disassembly systems with blocking and general distributions

A modelling methodology is presented for assembly/disassembly systems with general processing time distributions and finite buffers. The approach combines the distributions discretization and a decomposition technique to analyze large manufacturing systems in a reasonable computational time and with good accuracy. In the decomposition technique, the system is decomposed into two station subsystems and the processing time distributions of the virtual stations are iteratively modified to approximate the impact of the rest of the network, adding estimations of the blocking and starving distributions. To analyze each subsystem, the general processing time distributions are discretized by aggregation of the probability masses, and the subsystem is then analytically modeled using a discrete Markov chain. We first show that this approach allows an accurate estimation of the subsystems cycle time distributions, which is crucial in the decomposition technique. Using computational experiments, we show that our decomposition method leads to accurate performance evaluation for large manufacturing systems (relative error on the order of 1%) and that the fine distribution estimation indeed seems to bring an improvement. Furthermore, we show on examples that, using decomposition, the cycle time distributions can be approximated reliably for large systems.

Tractable Learning and Inference for Large-Scale Probabilistic Boolean Networks.

Probabilistic Boolean networks (PBNs) have previously been proposed so as to gain insights into complex dynamical systems. However, identification of large networks and their underlying discrete Markov chain which describes their temporal evolution still remains a challenge. In this paper, we introduce an equivalent representation for PBNs, the stochastic conjunctive normal form network (SCNFN), which enables a scalable learning algorithm and helps predict long-run dynamic behavior of large-scale systems. State-of-the-art methods turn out to be 400 times slower for middle-sized networks (i.e., containing 100 nodes) and incapable of terminating for large networks (i.e., containing 1000 nodes) compared to the SCNFN-based learning, when attempting to achieve comparable accuracy. In addition, in contrast to the currently used methods which introduce strict restrictions on the structure of the learned PBNs, the hypothesis space of our training paradigm is the set of all possible PBNs. Moreover, SCNFNs enable efficient sampling so as to statistically infer multistep transition probabilities which can provide information on the activity levels of individual nodes in the long run. Extensive experimental results showcase the scalability of the proposed approach both in terms of sample and runtime complexity. In addition, we provide examples to study large and complex cell signaling networks to show the potential of our model. Finally, we suggest several directions for future research on model variations, theoretical analysis, and potential applications of SCNFNs.

Dynamic indicator of intimate partner violence: self-regulatory perpetrator

A dynamic indicator to measure the level of violence between a perpetrator and victim of intimate partner violence is developed in this paper. Herein it is considered that violence can be measured through the rates of change over time of the violent interactions, considering the weight of severity of the violence committed. These rates of change are modelled by a logistic differential equation with a probabilistic parameter, the principle being the self-regulation of a perpetrator. This parameter of self-regulation is modelled by two changes of states: one of self-control and the other being a loss thereof. The transition of each state is modelled by a discrete Markov chain. With the proposed dynamic indicator, computational simulations are created to generate probable scenarios of partner violence with varying levels of severity. This dynamic indicator is then applied to a specific study of physical violence, using the prevalence of violence to approximate a state of loss of control. With the results obtained from the simulation, a qualitative analysis is made of the probable behaviours, controllable (semi-stable) and uncontrollable (potentially deadly), that can occur in a time interval of 3 and 12 months.

A Novel Capture-Aware TDMA-Based MAC Protocol for Safety Messages Broadcast in Vehicular Ad Hoc Networks

Since broadcast communication is considered the highly appropriate technique for the dissemination of safety messages in vehicular ad hoc networks (VANETs), it is vital to design a high-efficient medium access control (MAC) protocol for vehicles accessing wireless channel. However, the characteristics of VANETs, such as high-velocity vehicles, highly dynamic topology and very unstable communication link, lower the performance of existing MAC protocols. In this paper, we propose a novel Capture-aware TDMA-based MAC (CT-MAC) protocol, which can better utilize the channel resource than the existing MAC protocols by setting the optimal frame length with taking capture effect into account. To obtain the realistically optimal frame length, the closed form expression of probability of capture effect is derived under Nakagami-m fading channel which is proper to model the small-scale fading channel in VANETs. Besides, a discrete Markov chain is introduced to analyze the impact of capture effect on channel utilization. The theoretical analyses and the simulation results show that the reliability of broadcast and the efficiency of channel utilization are significantly improved.

On equalities of sigma-finite measures constructed from discrete recurrent Markov chains

On equalities of sigma-finite measures constructed from discrete recurrent Markov chains

Performance evaluation and analysis of a discrete queue system with multiple working vacations and non-preemptive priority

In this paper, we introduce a discrete time Geo/Geo/1 queue system with non-preemptive priority and multiple working vacations. We assume that there are two types of customers in this queue system named Customers of type-Ⅰ and Customers of type-Ⅱ. Customer of type-Ⅱ has a higher priority with non-preemption than Customer of type-Ⅰ. By building a discrete time four-dimensional Markov Chain which includes the numbers of customers with different priorities in the system, the state of the server and the service state, we obtain the state transition probability matrix. Using a birth-and-death chain and matrix-geometric method, we deduce the average queue length, the average waiting time of the two types of customers, and the average busy period of the system. Then, we provide some numerical results to evaluate the effect of the parameters on the system performance. Finally, we develop some benefit functions to analyse both the personal and social benefit, and obtain some optimization results within a certain range.

A stochastic model for intracellular active transport

We develop a stochastic model for an intracellular active transport problem. Our aims are to calculate the probability that a molecular motor reaches a hidden target, to study what influences this probability and to calculate the time required for the molecular motor to hit the target (Mean First Passage Time). We study different biologically relevant scenarios, which include the possibility of multiple hidden targets (which breed competition) and the presence of obstacles. The purpose of including obstacles is to illustrate actual disruptions of the intracellular transport (which can result, for example, in several neurological disorders. From a mathematical point of view, the intracellular active transport is modelled by two independent continuous-time, discrete space Markov chains: one for the dynamics of the molecular motor in the space intervals and one for the domain of target. The process is time homogeneous and independent of the position of the molecular motor.

Stochastic modeling of algal bloom dynamics with delayed nutrient recycling.

Using the discrete Markov chain, in this paper we develop a stochastic model for algal bloom, in which white noise terms are introduced to describe the effects of environmental random fluctuations and time delay to account for the time needed in the conversion of detritus into nutrient. For the proposed model, we firstly discuss the well-posedness, namely the existence and uniqueness of the global positive solution. Then, it is followed by seeking the sufficient conditions for the stochastic stability of its washout equilibrium. Then by using Fourier transform method, the spectral densities of the nutrient and the algae population are estimated. Finally, we show that larger noise can make the algae population extinct exponentially with probability one. Our theoretical and numerical results suggest that the environmental random fluctuations may have more significant influences on the dynamics of the model than the delay. These findings are helpful for a better understanding of the formation mechanism of algal blooms.

A stochastic model for active transport

We develop a stochastic model for an intracellular active transport problem. Our aims are to calculate the probability that a molecular motor reaches a hidden target, to study what influences this probability and to calculate the time required for the molecular motor to hit the target (mean first passage time). We study different biologically relevant scenarios, which include the possibility of multiple hidden targets (which breed competition) and the presence of obstacles. The purpose of including obstacles is to illustrate actual disruptions of the intracellular transport (which can result, for example, in several neurological disorders. From a mathematical point of view, the intracellular active transport is modelled by two independent continuous-time, discrete space Markov chains: one for the dynamics of the molecular motor in the space intervals and one for the domain of target. The process is time homogeneous and independent of the position of the molecular motor.

Computing the Bargaining Approach for Equalizing the Ratios of Maximal Gains in Continuous-Time Markov Chains Games

This paper presents a novel approach for computing the Kalai–Smorodinsky bargaining equilibrium for continuous time and discrete states Markov chains games. To solve the bargaining situation we set the disagreement point as the Nash equilibrium of the problem, then to find the new agreement point we follow the bargaining model presented by Kalai–Smorodinsky employing the utopia point concept. We exemplify the game formulation in terms of nonlinear programming equations implementing the Lagrange principle. The Tikhonov’s regularization method is applied to ensure the convergence of the cost-functions to an equilibrium point. For solving the problem we use a programming method implemented by the extraproximal optimization approach. The proposed method is validated by a numerical example related to the labor market problem for a three-person bargaining problem.

Cooperative spectrum sharing in cognitive radio networks with energy accumulation: design and analysis

The authors propose an efficient spectrum sharing scheme in cooperative cognitive radio networks, where an energy-constrained secondary transmitter (ST) first scavenges radio frequency (RF) energy from the received primary signals, and then the ST assists the primary transmission to obtain the opportunity of spectrum access. Specifically, the ST can forward the primary signal with its own signal by adopting both the Alamouti coding technique and superposition scheme only if the harvested energy is sufficient while the primary data is decoded correctly by the ST. Otherwise, the ST will continue to harvest RF energy. The authors use the discrete Markov chain to model the processes of charging and discharging of the battery. Moreover, two different joint decoding and interference cancellation schemes are employed at the receivers to restore the desired data. Closed-form expressions of outage probabilities for both the primary and secondary systems are derived. Aiming to minimise the outage probability of the secondary system with guaranteeing the primary transmission, an optimal power allocation factor for the ST is determined by Monte-Carlo simulation. Numerical results demonstrate that the proposed scheme can effectively improve the transfer performance of the secondary system while realising the transfer requirement of the primary system.

Why antioxidant therapies have failed in clinical trials

In spite of considerable research, and many clinical trials involving thousands of patients, there is a conspicuous lack of antioxidant therapies available. In this paper we present results for the interaction and neutralization of a free radical species. We adopt two modeling techniques, one based upon Gillespieâ;;s Stochastic Simulation Algorithm and one based upon a discrete Markov chain. An advantage of these models is that they incorporate the number of molecules present per unit volume, and with the Markov chain model, the relative dimensions of these molecules. By means of these models we question the basis of antioxidant therapies based on trapping or scavenging of reactive species. We demonstrate the extraordinary capacity of the enzymatic antioxidant defenses relative to non-enzymatic defenses. We conclude that, if the concentration of an non-enzymatic antioxidant is too low there is little chance of collision and interaction with a free radical species. Furthermore, if the rate of reaction between the free radical and the non-enzymatic antioxidant is below a necessary threshold then the effect of an antioxidant will be dwarfed by the free radical defense systems naturally present. As such we suggest that failure of most antioxidant therapies in clinical trials is to be expected.

Time-consistent risk-constrained dynamic portfolio optimization with transactional costs and time-dependent returns

Dynamic portfolio optimization has a vast literature exploring different simplifications by virtue of computational tractability of the problem. Previous works provide solution methods considering unrealistic assumptions, such as no transactional costs, small number of assets, specific choices of utility functions and oversimplified price dynamics. Other more realistic strategies use heuristic solution approaches to obtain suitable investment policies. In this work, we propose a time-consistent risk-constrained dynamic portfolio optimization model with transactional costs and Markovian time-dependence. The proposed model is efficiently solved using a Markov chained stochastic dual dynamic programming algorithm. We impose one-period conditional value-at-risk constraints, arguing that it is reasonable to assume that an investor knows how much he is willing to lose in a given period. In contrast to dynamic risk measures as the objective function, our time-consistent model has relatively complete recourse and a straightforward lower bound, considering a maximization problem. We use the proposed model for approximately solving: (i) an illustrative problem with 3 assets and 1 factor with an autoregressive dynamic; (ii) a high-dimensional problem with 100 assets and 5 factors following a discrete Markov chain. In both cases, we empirically show that our approximate solution is near-optimal for the original problem and significantly outperforms selected (heuristic) benchmarks. To the best of our knowledge, this is the first systematic approach for solving realistic time-consistent risk-constrained dynamic asset allocation problems.

Quantum Markov processes: From attractor structure to explicit forms of asymptotic states

Markov processes play an important role in physics and the theory of open systems in particular. In this paper we study the asymptotic evolution of trace-nonincreasing homogenous quantum Markov processes (both types, discrete quantum Markov chains and continues quantum dynamical semigroups) equipped with a so-called strictly positive T -state in the Schrodinger and the Heisenberg picture. We derive a fundamental theorem specifying the structure of the asymptotic and uncover a rich set of transformations between attractors of quantum Markov processes in both pictures. Moreover, we generalize the structure theorem derived for quantum Markov chains to quantum dynamical semigroups showing how the internal structure of generators of quantum Markov processes determines attractors in both pictures. Based on these results we provide two characterizations of all asymptotic and stationary states, both strongly reminding in form the well-known Gibbs states of statistical mechanics. We prove that the dynamics within the asymptotic space is of unitary type, i.e. quantum Markov processes preserve a certain scalar product of operators from the asymptotic space, but there is no corresponding unitary evolution on the original Hilbert space of pure states. Finally simple examples illustrating the derived theory are given.

Expanding the Markov Chain Toolbox: Distributions of Occupation Times and Waiting Times

Markov models are important tools for quantitative social research. In this article, new methods for discrete Markov chains are presented. These methods allow us to calculate the distribution of the occupation time in a subset of the state space, the distribution of the waiting time to first entry into a subset of the state space, and the distribution of the waiting time to final exit from a subset of the state space. To demonstrate the usefulness of these methods, we apply them to working life tables for Spanish males to assess how the recent financial crisis affected the length of working life. The results show that the duration of working life decreased considerably, a pattern that can largely be explained by later entry to and earlier exit from the labor market. The findings also indicate that inequality in the length of working life increased.

Single-Molecule, Real-Time Dissecting of Peptide Nucleic Acid-DNA Duplexes with a Protein Nanopore Tweezer.

Peptide nucleic acids (PNAs) are artificial, oligonucleotides analogues, where the sugar-phosphate backbone has been substituted with a peptide-like N-(2-aminoethyl)glycine backbone. Because of their inherent benefits, such as increased stability and enhanced binding affinity toward DNA or RNA substrates, PNAs are intensively studied and considered beneficial for the fields of materials and nanotechnology science. Herein, we designed cationic polypeptide-functionalized, 10-mer PNAs, and demonstrated the feasible detection of hybridization with short, complementary DNA substrates, following analytes interaction with the vestibule entry of an α-hemolysin (α-HL) nanopore. The opposite charged state at the polypeptide-functionalized PNA-DNA duplex extremities, facilitated unzipping of a captured duplex at the lumen entry of a voltage-biased nanopore, followed by monomers threading. These processes were resolvable and identifiable in real-time, from the temporal profile of the ionic current through a nanopore accompanying conformational changes of a single PNA-DNA duplex inside the α-HL nanopore. By employing a kinetic description within the discrete Markov chains theory, we proposed a minimalist kinetic model to successfully describe the electric force-induced strand separation in the duplex. The distinct interactions of the duplex at either end of the nanopore present powerful opportunities for introducing new generations of force-spectroscopy nanopore-based platforms, enabling from the same experiment duplex detection and assessment of interstrand base pairing energy.

On the reduction property of the number of H-equivalent tuples of states in a discrete Markov chain

Abstract The phenomenon of reduction of the set of permutations H arising in theorems on the weak convergence of the number of pairs of H-equivalent tuples in a segment of an indecomposable finite Markov chain to discrete distributions of the Poisson type is investigated.

Asynchronous Device Detection for Cognitive Device-to-Device Communications

Dynamic spectrum sharing will facilitate the interference coordination in device-to-device (D2D) communications. In the absence of network level coordination, the timing synchronization among D2D users will be unavailable, leading to inaccurate channel state estimation and device detection, especially in time-varying fading environments. In this paper, we design an asynchronous device detection/discovery framework for cognitive-D2D applications, which acquires timing drifts and dynamical fading channels when directly detecting the existence of a proximity D2D device (e.g. or primary user). To model and analyze this, a new dynamical system model is established, where the unknown timing deviation follows a random process, while the fading channel is governed by a discrete state Markov chain. To cope with the mixed estimation and detection problem, a novel sequential estimation scheme is proposed, using the conceptions of statistic Bayesian inference and random finite set. By tracking the unknown states (i.e. varying time deviations and fading gains) and suppressing the link uncertainty, the proposed scheme can effectively enhance the detection performance. The general framework, as a complimentary to a network-aided case with the coordinated signaling, provides the foundation for development of flexible D2D communications along with proximity-based spectrum sharing.