Markov Property Research Articles

The impact of pinning points on memorylessness in Lévy random bridges

Abstract Random bridges have gained significant attention in recent years due to their potential applications in various areas, particularly in information-based asset pricing models. This paper aims to explore the potential influence of the pinning point’s distribution on the memorylessness and stochastic dynamics of the bridge process. We introduce Lévy bridges with random length and random pinning points, and analyze their Markov property. Our study demonstrates that the Markov property of Lévy bridges depends on the nature of the distribution of their pinning points. The law of any random variables can be decomposed into singular continuous, discrete, and absolutely continuous parts with respect to the Lebesgue measure (Lebesgue’s decomposition theorem). We show that the Markov property holds when the pinning points’ law does not have an absolutely continuous part. Conversely, the Lévy bridge fails to exhibit Markovian behavior when the pinning point has an absolutely continuous part.

Generalised Entropy Accumulation

Consider a sequential process in which each step outputs a system Ai\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$A_i$$\\end{document} and updates a side information register E. We prove that if this process satisfies a natural “non-signalling” condition between past outputs and future side information, the min-entropy of the outputs A1,⋯,An\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$A_1, \\dots , A_n$$\\end{document} conditioned on the side information E at the end of the process can be bounded from below by a sum of von Neumann entropies associated with the individual steps. This is a generalisation of the entropy accumulation theorem (EAT) (Dupuis et al. in Commun Math Phys 379: 867–913, 2020), which deals with a more restrictive model of side information: there, past side information cannot be updated in subsequent rounds, and newly generated side information has to satisfy a Markov condition. Due to its more general model of side-information, our generalised EAT can be applied more easily and to a broader range of cryptographic protocols. As examples, we give the first multi-round security proof for blind randomness expansion and a simplified analysis of the E91 QKD protocol. The proof of our generalised EAT relies on a new variant of Uhlmann’s theorem and new chain rules for the Rényi divergence and entropy, which might be of independent interest.

Time for memorable consumption

A consumption event is memorable if the memory of the event affects well-being at times after the material consumption, as originally introduced by Gilboa et al. (2016). Our main contribution is to develop an axiomatic foundation of memorable consumption in a dynamic setting. Preferences are represented by the present value of the sum of utilities derived at each date from the current consumption and from recollecting the past. Our model accommodates well-known phenomena in psychology, such as the peak-end rule, duration neglect, and adaptation trends. We also provide foundations for a prominent special case of the representation with the Markovian property. The model is illustrated with applications in two different contexts: risk-taking behavior in a principal-agent problem and life-cycle consumption-savings decisions.

Mixed orthogonality graphs for continuous-time stationary processes

In this paper, we introduce different concepts of Granger causality and contemporaneous correlation for multivariate stationary continuous-time processes to model different dependencies between the component processes. Several equivalent characterisations are given for the different definitions, in particular by orthogonal projections. We then define two mixed graphs based on different definitions of Granger causality and contemporaneous correlation, the (mixed) orthogonality graph and the local (mixed) orthogonality graph. In these graphs, the components of the process are represented by vertices, directed edges between the vertices visualise Granger causal influences and undirected edges visualise contemporaneous correlation between the component processes. Further, we introduce various notions of Markov properties in analogy to Eichler (2012), which relate paths in the graphs to different dependence structures of subprocesses, and we derive sufficient criteria for the (local) orthogonality graph to satisfy them. Finally, as an example, for the popular multivariate continuous-time AR (MCAR) processes, we explicitly characterise the edges in the (local) orthogonality graph by the model parameters.

A Stochastic Approach in the Analysis of Compartmental Models in Epidemiology

The spread of infectious diseases is a significant public health issue. Mathematical modeling is being used to find appropriate answers to this question. In this paper we propose a stochastic approach to compartmental models. A new model allows to rely on the fokker-planck equation with a SEIR model, with constant rates, to take into account the random environment in the propagation of a disease, markov and kolmogorov properties have been used in the modeling of the process. We have also determined the basic reproduction rate R0 of our new stochastic model using the new generation matrix.

Markovian lifting and asymptotic log-Harnack inequality for stochastic Volterra integral equations

We introduce a new framework of Markovian lifts of stochastic Volterra integral equations (SVIEs for short) with completely monotone kernels. We define the state space of the Markovian lift as a separable Hilbert space which incorporates the singularity or regularity of the kernel into the definition. We show that the solution of an SVIE is represented by the solution of a lifted stochastic evolution equation (SEE for short) defined on the Hilbert space and prove the existence, uniqueness and Markov property of the solution of the lifted SEE. Furthermore, we establish an asymptotic log-Harnack inequality and some consequent properties for the Markov semigroup associated with the Markovian lift via the asymptotic coupling method.

Extending a rational process model of causal reasoning: Assessing Markov violations and explaining away with inhibitory causal relations.

Causes sometimes decrease rather increase the probability of an effect, as when drinking coffee lowers the probability of sleep or an aspirin eliminates a headache. This research tests how two causal reasoning errors that have influenced the development of theories of human causal reasoning manifest themselves in the presence of inhibitory causal relations. Past research with generative causal relations (a cause makes its effect more probable) has shown that people violate the Markov condition, the pattern of independence that should obtain among causally related variables. And it has shown that they explain away-the phenomenon in which one should lower likelihood of one event when another is discovered to have occurred (e.g., exonerating one murder suspect when evidence against another is found)-too little or not at all. The new empirical findings reported here reveal that both sorts of errors manifest themselves when inhibitory causal relations are present although, unexpectedly, the direction of those errors sometimes reverses. Only the mutation sampler, a rational process model of human causal reasoning, correctly predicted these novel empirical findings. These results support the view that causal reasoning errors can be understood as arising from rational inference constrained by limited cognitive resources. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Model-free inference of memory in conformational dynamics of a multi-domain protein

Abstract Single-molecule experiments provide insight into the motion (conformational dynamics) of individual protein molecules. Usually, a well-defined but coarse-grained intramolecular coordinate is measured and subsequently analysed with the help of hidden Markov models to deduce the kinetics of protein conformational changes. Such approaches rely on the assumption that the microscopic dynamics of the protein evolve according to a Markov-jump process on some network. However, the manifestation and extent of memory in the dynamics of the observable strongly depends on the chosen underlying Markov model, which is generally not known and therefore can lead to misinterpretations. Here, we combine extensive single-molecule plasmon ruler experiments on the heat shock protein Hsp90, computer simulations, and theory to infer and quantify memory in a model-free fashion. Our analysis is based on the bare definition of non-Markovian behaviour and does not require any underlying model. In the case of Hsp90 probed by a plasmon ruler, the Markov assumption is found to be clearly and conclusively violated on timescales up to roughly 50 s, which corresponds roughly to ∼50% of the inferred correlation time of the signal. The extent of memory is striking and reaches biologically relevant timescales. This implies that memory effects penetrate even the slowest observed motions. We provide clear and reproducible guidelines on how to test for the presence and duration of memory in experimental single-molecule data.

Testing for Markovian character of transfer of fluctuations in solar wind turbulence on kinetic scales.

We apply statistical analysis to search for processes responsible for turbulence in physical systems. In our previous studies, we have shown that solar wind turbulence in the inertial range of large magnetohydrodynamic scales exhibits Markov properties. We have recently extended this approach on much smaller kinetic scales. Here we are testing for the Markovian character of stochastic processes in a kinetic regime based on magnetic field and velocity fluctuations in the solar wind, measured onboard the Magnetospheric Multiscale (MMS) mission: behind the bow shock, inside the magnetosheath, and near the magnetopause. We have verified that the Chapman-Kolmogorov necessary conditions for Markov processes is satisfied for local transfer of energy between the magnetic and velocity fields also on kinetic scales. We have confirmed that for magnetic fluctuations, the first Kramers-Moyal coefficient is linear, while the second term is quadratic, corresponding to drift and diffusion processes in the resulting Fokker-Planck equation. It means that magnetic self-similar turbulence is described by generalized Ornstein-Uhlenbeck processes. We show that for the magnetic case, the Fokker-Planck equationleads to the probability density functions of the kappa distributions, which exhibit global universal scale invariance with a linear scaling and lack of intermittency. On the contrary, for velocity fluctuations, higher order Kramers-Moyal coefficients should be taken into account and hence scale invariance is not observed. However, the nonextensity parameter in Tsallis entropy provides a robust measure of the departure of the system from equilibrium. The obtained results are important for a better understanding of the physical mechanism governing turbulent systems in space and laboratory.

The most likely common cause

The common cause principle for two random variables A and B is examined in the case of causal insufficiency, when their common cause C is known to exist, but only the joint probability of A and B is observed. As a result, C cannot be uniquely identified (the latent confounder problem). We show that the generalized maximum likelihood method can be applied to this situation and allows identification of C that is consistent with the common cause principle. It closely relates to the maximum entropy principle. Investigation of the two binary symmetric variables reveals a non-analytic behavior of conditional probabilities reminiscent of a second-order phase transition. This occurs during the transition from correlation to anti-correlation in the observed probability distribution. The relation between the generalized likelihood approach and alternative methods, such as predictive likelihood and minimum common entropy, is discussed. The consideration of the common cause for three observed variables (and one hidden cause) uncovers causal structures that defy representation through directed acyclic graphs with the Markov condition.

Multi-hazard life-cycle consequence analysis of deteriorating engineering systems

Probabilistic life-cycle consequence (LCCon) analysis (e.g., assessment of repair costs, downtime, or casualties over an asset’s service life) can enable optimal life-cycle management of critical assets under uncertainties. This can lead to effective risk-informed decision-making for future disaster management (i.e., risk mitigation and/or resilience-enhancing strategies/policies) implementation. Nevertheless, despite recent advances in understanding, modeling, and quantifying multiple-hazard (or multi-hazard) interactions, most available LCCon analytical formulations fail to accurately compute the exacerbated consequences which may stem from incomplete or absent repair actions between different interacting hazard events. This paper introduces a discrete-time, discrete-state Markovian framework for efficient multi-hazard LCCon analysis of deteriorating engineering systems (e.g., buildings, infrastructure components) that appropriately accounts for complex interactions between natural hazard events and their effects on a system’s performance. The Markovian assumption is used to model the probability of a system being in any performance level (i.e., limit state) after multiple hazard events inducing either instantaneous and/or gradual deterioration and after potential repair actions through implementing stochastic (transition) matrices. LCCon estimates are then obtained by combining limit state probabilties with suitable system-level consequence models in a computationally efficient manner. The proposed framework is illustrated for two case studies subject to earthquake and flood events as well as environment-induced corrosion during their service life. The first is a reinforced concrete building and the second is a simple transportation road network with a reinforced concrete bridge.

Recovering Complete Positivity of Non-Markovian Quantum Dynamics with Choi-Proximity Regularization

A relevant problem in the theory of open quantum systems is the lack of complete positivity of dynamical maps obtained after weak-coupling approximations, a famous example being the Redfield master equation. A number of approaches exist to recover well-defined evolutions under additional Markovian assumptions, but much less is known beyond this regime. Here, we propose a numerical method to cure the complete-positivity violation issue while preserving the non-Markovian features of an arbitrary original dynamical map. The idea is to replace its unphysical Choi operator with its closest physical one, mimicking recent work on quantum process tomography. We also show that the regularized dynamics is more accurate in terms of reproducing the exact dynamics, which allows us to heuristically push the utilization of these master equations in moderate coupling regimes, where the loss of positivity can have a relevant impact. Published by the American Physical Society 2024

Markov conditions and factorization in logical credal networks

We examine the recently proposed language of Logical Credal Networks, a powerful representation formalism that combines probabilities and logic. In particular we investigate the consequences of distinct Markov conditions upon their underlying semantics. We introduce the notion of structure for a Logical Credal Network and show that a structure without directed cycles leads to a well-known factorization result. For networks with directed cycles, we discuss the differences between Markov conditions, factorization results, and specification requirements. We consider several scenarios in causal reasoning that can be tackled by the formalism, in particular looking at partial identifiability and cycles.

Temporal dynamics unleashed: Elevating variational graph attention

This research introduces the Variational Graph Attention Dynamics (VarGATDyn), addressing the complexities of dynamic graph representation learning, where existing models, tailored for static graphs, prove inadequate. VarGATDyn melds attention mechanisms with a Markovian assumption to surpass the challenges of maintaining temporal consistency and the extensive dataset requirements typical of RNN-based frameworks. It harnesses the strengths of the Variational Graph Auto-Encoder (VGAE) framework, Graph Attention Networks (GAT), and Gaussian Mixture Models (GMM) to adeptly navigate the temporal and structural intricacies of dynamic graphs. Through the strategic application of GMMs, the model handles multimodal patterns, thereby rectifying misalignments between prior and estimated posterior distributions. An innovative multiple-learning methodology bolsters the model’s adaptability, leading to an encompassing and effective learning process. Empirical tests underscore VarGATDyn’s dominance in dynamic link prediction across various datasets, highlighting its proficiency in capturing multimodal distributions and temporal dynamics.

Mbtransfer: Microbiome intervention analysis using transfer functions and mirror statistics.

Time series studies of microbiome interventions provide valuable data about microbial ecosystem structure. Unfortunately, existing models of microbial community dynamics have limited temporal memory and expressivity, relying on Markov or linearity assumptions. To address this, we introduce a new class of models based on transfer functions. These models learn impulse responses, capturing the potentially delayed effects of environmental changes on the microbial community. This allows us to simulate trajectories under hypothetical interventions and select significantly perturbed taxa with False Discovery Rate guarantees. Through simulations, we show that our approach effectively reduces forecasting errors compared to strong baselines and accurately pinpoints taxa of interest. Our case studies highlight the interpretability of the resulting differential response trajectories. An R package, mbtransfer, and notebooks to replicate the simulation and case studies are provided.

Limited data on infectious disease distribution exposes ambiguity in epidemic modeling choices

Traditional disease transmission models assume that the infectious period is exponentially distributed with a recovery rate fixed in time and across individuals. This assumption provides analytical and computational advantages, however, it is often unrealistic when compared to empirical data. Current efforts in modeling nonexponentially distributed infectious periods are either limited to special cases or lead to unsolvable models. Also, the link between empirical data (the infectious period distribution) and the modeling needs (the definition of the corresponding recovery rates) lacks a clear understanding. Here we introduce a mapping of an arbitrary distribution of infectious periods into a distribution of recovery rates. Under the Markovian assumption to ensure analytical tractability, we show that the same infectious period distribution at the population level can be reproduced by two modeling schemes that we call and , depending on the individual response to the infection, and aggregated empirical data cannot easily discriminate the correct scheme. Besides being conceptually different, the two schemes also lead to different epidemic trajectories. Although sharing the same behavior close to the disease-free equilibrium, the scheme deviates from the expected epidemic when reaching the endemic equilibrium of a susceptible-infectious-susceptible transmission model, while the scheme turns out to be equivalent to assuming a homogeneous recovery rate. We show this through analytical computations and stochastic epidemic simulations on a contact network, using both generative network models and empirical contact data. It is therefore possible to reproduce heterogeneous infectious periods in network-based transmission models, however, the resulting prevalence is sensitive to the modeling choice for the interpretation of the empirically collected data on the length of the infectious period. In the absence of higher resolution data, studies should acknowledge such deviations in the epidemic predictions. Published by the American Physical Society 2024

A fast wide-range air balancing control method based on deep reinforcement learning

Air balancing is a crucial technology for reducing energy consumption in ventilation ducts and improving indoor environmental quality. Existing air balancing methods face challenges due to the complex and diverse internal duct topologies of ventilation duct systems, as well as the strong coupling of airflow between branches, making air balancing tuning difficult. To address the slow convergence speed and difficulty in precise control of air balancing in multi-branch ventilation systems, this paper proposes a fast wide-range air balancing control method based on deep reinforcement learning (DRL-AB). A system airflow control process with Markovian properties is designed, with state, action, and reward functions oriented towards reducing airflow errors. A dynamic single-branch target training mechanism is designed to reduce training costs while improving the agent’s training efficiency and its ability to generalize to different target airflow levels. The proposed method’s performance under various target airflow conditions is verified through experimental platforms. The results indicate that, in all test cases, the maximum percentage error is controlled within 3.33%, ensuring rapid and accurate convergence of a wide range of target airflows. This demonstrates strong universality, prevents waste of airflow energy, and enhances energy utilization efficiency.

A recurrent reinforcement learning strategy for optimal scheduling of partially observable job-shop and flow-shop batch chemical plants under uncertainty

This study presents a methodology that makes use of Deep Recurrent Q-Learning to develop an agent that acts as an online scheduler for flow-shop or job-shop batch plants with zero-wait restriction under uncertainty. The environment is assumed to be partially observable, i.e., it does not follow the Markov property and information has to be gathered from previous time intervals. The processing times of the machines are unknown to the agent whereas production demand realizations are provided during the operation and not known a priori. The agent aims to complete the demands and to minimize the makespan of the process. Moreover, the agent should avoid violation of constraints associated with product allocation, time horizon, and storage capacity. Three case studies featuring two job-shops and one flow-shop are presented to show the benefits of this framework with environments where the information is limited. Results showed that the agents can generate schedules considering the uncertain parameters of the system while aiming to reduce the makespan of the process. Tests on the agents resulted in small errors in the decision-making process (less than 2%) thus demonstrating that DRQN can serve as a reliable tool for online scheduling subject to uncertainty in partially observable job and flow-shops.

Checkpointing models for tasks of different types

A server subject to random breakdowns and repairs offers services to incoming jobs whose lengths are highly variable. A checkpointing policy is in operation, aiming to protect against possibly lengthy recovery periods by backing up the current state at periodic checkpoints. The problem of how to choose a checkpointing interval to optimise performance is addressed by analysing a general queueing model which includes breakdowns, repairs, back-ups and recoveries. Exact solutions are obtained under both Markovian and non-Markovian assumptions. Numerical experiments illustrate the conditions where checkpoints are useful and where they are not, and, in the former case, quantify the achievable benefits.

Distributionally Robust Observable Strategic Queues

This paper presents an extension of Naor’s analysis on the join-or-balk problem in observable M/M/1 queues. Although all other Markovian assumptions still hold, we explore this problem assuming uncertain arrival rates under the distributionally robust settings. We first study the problem with the classical moment ambiguity set, where the support, mean, and mean-absolute deviation of the underlying distribution are known. Next, we extend the model to the data-driven setting, where decision makers only have access to a finite set of samples. We develop three optimal joining threshold strategies from the perspectives of an individual customer, a social optimizer, and a revenue maximizer such that their respective worst-case expected benefit rates are maximized. Finally, we compare our findings with Naor’s original results and the traditional sample average approximation scheme. Funding: This research was supported by the National Science Foundation [Grants 2342505 and 2343869].