Classical Markov Research Articles

Quantum Bayesian Inference with Renormalization for Gravitational Waves

Abstract Advancements in gravitational-wave (GW) interferometers, particularly the next generation, are poised to enable the detections of orders of magnitude more GWs from compact binary coalescences. While the surge in detections will profoundly advance GW astronomy and multimessenger astrophysics, it also poses significant computational challenges in parameter estimation. In this work, we introduce a hybrid quantum algorithm qBIRD, which performs quantum Bayesian inference with renormalization and downsampling to infer GW parameters. We validate the algorithm using both simulated and observed GWs from binary black hole mergers on quantum simulators, demonstrating that its accuracy is comparable to classical Markov Chain Monte Carlo methods. Currently, our analyses focus on a subset of parameters, including chirp mass and mass ratio, due to the limitations from classical hardware in simulating quantum algorithms. However, qBIRD can accommodate a broader parameter space when the constraints are eliminated with a small-scale quantum computer of sufficient logical qubits.

Estimating Extreme Drought Risk Through Classical and Bayesian Paradigms

ABSTRACTDrought poses significant challenges to both the environment and the economy, necessitating proactive mitigation strategies. This study introduces both classical and Bayesian Markov Chain Monte Carlo (MCMC) extreme value probabilistic models for quantifying drought risk. The models utilise the generalised extreme value (GEV) distribution to characterise the distribution of standardised precipitation index (SPI) and non‐stationary standardised precipitation index (NSSPI) variables. Drought risk is probabilistically assessed across five regions in Baluchistan (a drought‐prone area of Pakistan) over two 20‐year periods per region. The study presents a novel approach in probabilistic quantification models, demonstrating slight performance improvement with the Bayesian MCMC paradigm, as evaluated by the continuously ranked probability scoring. Moreover, the application of the presented methodology can be extended to other climatic zones using Bayesian MCMC with informative priors constructed from historical records of the neighbouring regions.

Quantum Exclusion Process driven by Brownian Motions in terms of quantum Bernoulli noises

Quantum Bernoulli noises are the family of annihilation and creation operators on Bernoulli functionals, which satisfy a canonical anti-commutation relation in equal time. The paper is devoted to the study of Quantum Exclusion Process driven by Brownian Motions in terms of Quantum Bernoulli Noises. We first study the existence and uniqueness of regular solutions to the classical Quantum Exclusion Process driven by Brownian Motions. We then provide an explicit representation formula by separating the action on off-diagonal and diagonal operators, on which they are reduced to the Quantum Exclusion Process of classical Markov chains. Finally, we prove that the existence of regular invariant measures for Quantum Exclusion Process driven by Brownian Motions in terms of Quantum Bernoulli Noises.

Review of Continuous Time Markov Bridges: Models, Algorithms and Applications

Abstract. Continuous time Markov chains (CTMC) with start point and endpoint, i.e., a continuous time Markov Bridge, have become an important mathematical model in a wide variety of scientific disciplines and simulations. In this research, we review some classical Markov Bridge simulation algorithms and Markov Bridge's mathematical model, Kinetic Monte Carlo, Direct Sampling, Reject Sampling and Uniform Sampling with a potential application algorithm in biological protein folding evolution problem.

Exploring spatiotemporal dynamics, seasonality, and time-of-day trends of PM2.5 pollution with a low-cost sensor network: Insights from classic and spatially explicit Markov chains

Fine particulate matter (PM2.5) is a major health and environmental concern, with significant spatiotemporal dynamics in urban areas. Low-cost air quality sensor (LCS) networks offer a paradigm-changing opportunity to acquire high spatiotemporal resolution data, revealing the urban pollution landscape with sufficient detail for effective policymaking and health assessment. This study advances geospatial air quality research by using classic and spatial Markov chains to analyze the seasonality and intra-daily variations of PM2.5 using LCS data. Results highlight distinctive PM2.5 seasonality, with the “Good” state predominating in summer and being least common in winter. Midday is the peak period for the “Good” state, while mornings and nights have poorer conditions, suggesting a need for stricter pollution control during evening traffic rush hours. Notably, the impact of temporal scale on spatial Markov analysis is substantial, showing a broader range of air pollution states, increased stability, and reduced variation between time intervals compared to daily assessments. Site-level analysis reveals that rural sites are more likely to maintain “Good” state and less likely to transition out of it. Overall, this study highlights the effectiveness of high spatiotemporal resolution data and demonstrates the capacity of Markov chains to reveal nuances in phenomena such as air pollution.

Quantum fluctuation dynamics of open quantum systems with collective operator-valued rates, and applications to Hopfield-like networks

We consider a class of open quantum many-body systems that evolves in a Markovian fashion, the dynamical generator being in GKS-Lindblad form. Here, the Hamiltonian contribution is characterized by an all-to-all coupling, and the dissipation features local transitions that depend on collective, operator-valued rates, encoding average properties of the system. These types of generators can be formally obtained by generalizing, to the quantum realm, classical (mean-field) stochastic Markov dynamics, with state-dependent transitions. Focusing on the dynamics emerging in the limit of infinitely large systems, we build on the exactness of the mean-field equations for the dynamics of average operators. In this framework, we derive the dynamics of quantum fluctuation operators, that can be used in turn to understand the fate of quantum correlations in the system. We then apply our results to quantum generalized Hopfield associative memories. Here we show that, asymptotically and at the description level of quantum fluctuations, only a very weak amount of quantum correlations, in the form of quantum discord, emerges beyond classical correlations.

On a Mixed Transient–Asymptotic Result for the Sequential Interval Reliability for Semi-Markov Chains

In this paper, we are concerned with the study of sequential interval reliability, a measure recently introduced in the literature. This measure represents the probability of the system working during a sequence of nonoverlapping time intervals. In the cited work, the authors proposed a recurrent-type formula for computing this indicator in the transient case and investigated the asymptotic behavior as all the time intervals go to infinity. The purpose of the present work is to further explore the asymptotic behavior when only some of the time intervals are allowed to go to infinity while the remaining ones are not. In this way, we provide a unique indicator that is able to describe the process evolution in the transient and asymptotic cases as well. It is important to mention that this is not a straightforward result since, in order to achieve it, we need to develop several mathematical ingredients that generalize the classical renewal and Markov renewal frameworks. A numerical example illustrates our theoretical results.

Fault-tolerant control for Markov jump nonlinear systems based on an observer and finite-time fast integral terminal sliding mode control

This paper develops a novel fault-tolerant control based on an observer for underactuated robot manipulators with unknown external disturbances, uncertainties, and actuator faults. First, unlike the traditional approach of treating the robot system as a parameter time-varying system, the underactuated robot manipulator with different drive modes can be considered as one of the classical Markov jump nonlinear systems (MJNSs). Second, an adaptive disturbance observer is designed to estimate the state and disturbances of the system. Finally, based on the observation results, a nonsingular fast integral terminal sliding mode controller (NFITSMC) is utilized to implement fault-tolerant control of the system. Compared with traditional observer and terminal sliding mode controller, the adoption of the novel controller and observer can improve response time and reduce chattering. Especially, in order to eliminate chattering, the integral term is introduced into the nonsingular fast terminal sliding mode controller. Structuring Lyapunov–Krasovskii functional (LKF) and based on linear matrix inequalities (LMIs) techniques, the convergence of control strategy and tracking errors is proved. The simulation results show that the actuator faults can be observed successfully and the error system is finite-time stable.

The structure of the local time of Markov processes indexed by Lévy trees

We construct an additive functional playing the role of the local time—at a fixed point x—for Markov processes indexed by Lévy trees. We start by proving that Markov processes indexed by Lévy trees satisfy a special Markov property which can be thought as a spatial version of the classical Markov property. Then, we construct our additive functional by an approximation procedure and we characterize the support of its Lebesgue-Stieltjes measure. We also give an equivalent construction in terms of a special family of exit local times. Finally, combining these results, we show that the points at which the Markov process takes the value x encode a new Lévy tree and we construct explicitly its height process. In particular, we recover a recent result of Le Gall concerning the subordinate tree of the Brownian tree where the subordination function is given by the past maximum process of Brownian motion indexed by the Brownian tree.

Criteria for Davies irreducibility of Markovian quantum dynamics

The dynamics of Markovian open quantum systems are described by Lindblad master equations, generating a quantum dynamical semigroup. An important concept for such systems is (Davies) irreducibility, i.e. the question whether there exist non-trivial invariant subspaces. Steady states of irreducible systems are unique and faithful, i.e. they have full rank. In the 1970s, Frigerio showed that a system is irreducible if the Lindblad operators span a self-adjoint set with trivial commutant. We discuss a more general and powerful algebraic criterion, showing that a system is irreducible if and only if the multiplicative algebra generated by the Lindblad operators La and the operator iH+∑aLa†La , involving the Hamiltonian H, is the entire operator space. Examples for two-level systems, show that a change of Hamiltonian terms as well as the addition or removal of dissipators can render a reducible system irreducible and vice versa. Examples for many-body systems show that a large class of spin chains can be rendered irreducible by dissipators on just one or two sites. Additionally, we discuss the decisive differences between (Davies) reducibility and Evans reducibility for quantum channels and dynamical semigroups which has lead to some confusion in the recent physics literature, especially, in the context of boundary-driven systems. We give a criterion for quantum reducibility in terms of associated classical Markov processes and, lastly, discuss the relation of the main result to the stabilization of pure states and argue that systems with local Lindblad operators cannot stabilize pure Fermi-sea states.

Unveiling hidden companions in post-common-envelope binaries: A robust strategy and uncertainty exploration

Context. Some post-common-envelope binaries (PCEBs) are binary stars with short periods that exhibit significant period variations over long observational time spans. These eclipse timing variations (ETVs) are most likely to be accounted for by the presence of an unseen massive companion, potentially of planetary or substellar nature, and the light-travel time (LTT) effect. The existence of such companions challenges our current understanding of planetary formation and stellar evolution. Aims. In this study, our main objective is to describe the diversity of compatible nontransit companions around PCEBs and explore the robustness of the solutions by employing tools for uncertainty estimation. We select the controversial data of the QS Vir binary star, which previous studies have suggested hosts a planet. Methods. We employ a minimizing strategy, using genetic algorithms to explore the global parameter space followed by refinement of the solution using the simplex method. We evaluate errors through the classical Markov chain Monte Carlo (MCMC) approach and discuss the error range for parameters, considering the 1σ values obtained from the minimization. Results. Our results highlight the strong dependence of ETV models for close binaries on the dataset used, which leads to relatively loose constraints on the parameters of the unseen companion. We find that the shape of the O – C curve is influenced by the dataset employed. We propose an alternative method to evaluate errors on the orbital fits based on a grid search surrounding the best-fit values, obtaining a wider range of plausible solutions that are compatible with goodness-of-fit statistics. We also analyze how the parameter solutions are affected by the choice of the dataset, and find that this system continuously changes the compatible solutions as new data are obtained from eclipses. Conclusions. The best-fit parameters for QS Vir correspond to a low-mass stellar companion (57.71 Mjup ranging from ~40 to ~64 Mjup) on an eccentric orbit (e = 0.91−0.17+0.07) with a variety of potential periods (P = 16.69−0.42+0.47 yr.). Most solutions within 1σ exhibit regular orbits, despite their high eccentricity. Additional observations are required to accurately determine the period and other parameters of the unseen companion. In this context, we propose that a fourth body should not be modeled to fit the data, unless new observations considerably modify the computed orbital parameters. This methodology can be applied to other evolved binary stars suspected of hosting companions.

Quantum physics cannot be captured by classical linear hidden variable theories even in the absence of entanglement

Recent experimental tests of Bell inequalities confirm that entangled quantum systems cannot be described by local classical theories but still do not answer the question whether or not quantum systems could, in principle, be modeled by linear hidden variable theories. In this paper, we study the quantum trajectories of a single qubit that experiences a sequence of repeated generalized measurements. It is shown that this system, which constitutes a hidden quantum Markov model, is more likely to produce complex time correlations than any classical hidden Markov model with two output symbols. From this, we conclude that quantum physics cannot be replaced by linear hidden variable theories. Indeed, it has already been recognized that not only entanglement but also non-classical time correlations of quantum systems with quantum feedback are a valuable resource for quantum technology applications.

Quantum Theory in Finite Dimension Cannot Explain Every General Process with Finite Memory

Arguably, the largest class of stochastic processes generated by means of a finite memory consists of those that are sequences of observations produced by sequential measurements in a suitable generalized probabilistic theory (GPT). These are constructed from a finite-dimensional memory evolving under a set of possible linear maps, and with probabilities of outcomes determined by linear functions of the memory state. Examples of such models are given by classical hidden Markov processes, where the memory state is a probability distribution, and at each step it evolves according to a non-negative matrix, and hidden quantum Markov processes, where the memory is a finite-dimensional quantum system, and at each step it evolves according to a completely positive map. Here we show that the set of processes admitting a finite-dimensional explanation do not need to be explainable in terms of either classical probability or quantum mechanics. To wit, we exhibit families of processes that have a finite-dimensional explanation, defined manifestly by the dynamics of an explicitly given GPT, but that do not admit a quantum, and therefore not even classical, explanation in finite dimension. Furthermore, we present a family of quantum processes on qubits and qutrits that do not admit a classical finite-dimensional realization, which includes examples introduced earlier by Fox, Rubin, Dharmadikari and Nadkarni as functions of infinite-dimensional Markov chains, and lower bound the size of the memory of a classical model realizing a noisy version of the qubit processes.

A new quantum machine learning algorithm: split hidden quantum Markov model inspired by quantum conditional master equation

The Hidden Quantum Markov Model (HQMM) has significant potential for analyzing time-series data and studying stochastic processes in the quantum domain as an upgrading option with potential advantages over classical Markov models. In this paper, we introduced the split HQMM (SHQMM) for implementing the hidden quantum Markov process, utilizing the conditional master equation with a fine balance condition to demonstrate the interconnections among the internal states of the quantum system. The experimental results suggest that our model outperforms previous models in terms of scope of applications and robustness. Additionally, we establish a new learning algorithm to solve parameters in HQMM by relating the quantum conditional master equation to the HQMM. Finally, our study provides clear evidence that the quantum transport system can be considered a physical representation of HQMM. The SHQMM with accompanying algorithms present a novel method to analyze quantum systems and time series grounded in physical implementation.

Extremals in the Markov – Dubins Problem with Control on a Triangle

We formulate a time-optimal problem for a differential drive robot with bounded positive velocities of the driving wheels. This problem is equivalent to a generalization of the classical Markov – Dubins problem with an extended domain of control. We classify all extremal controls via the Pontryagin maximum principle. Some optimality conditions are obtained; therefore, the optimal synthesis is reduced to the enumeration of a finite number of possible solutions.

The Mixture Transition Distribution approach to networks: Evidence from stock markets

Networks can be built by using correlations between time series. The approach based on correlations has many advantages which are essentially related to its simplicity. Nevertheless, it is well known that time series may show strong dependence even if they are uncorrelated. In this paper, we will advance a multivariate Markov chain model based on the Mixture Transition Distribution (MTD) model to build networks between time series. The multivariate MTD is able to consider the dependence between time series and, at the same time, reduce the number of parameters to be estimated compared to the classical multivariate Markov chain. We show, by a numerical example, that the multivariate MTD outperforms the classical correlation approach. Moreover, using the same model, we build the network of the 30 constituents of the Dow Jones index showing the usefulness of the methodology in real problems in financial markets.

A Gibbs-INLA algorithm for multidimensional graded response model analysis.

In this paper, we propose a novel Gibbs-INLA algorithm for the Bayesian inference of graded response models with ordinal response based on multidimensional item response theory. With the combination of the Gibbs sampling and the integrated nested Laplace approximation (INLA), the new framework avoids the cumbersome tuning which is inevitable in classical Markov chain Monte Carlo (MCMC) algorithm, and has low computing memory, high computational efficiency with much fewer iterations, and still achieve higher estimation accuracy. Therefore, it has the ability to handle large amount of multidimensional response data with different item responses. Simulation studies are conducted to compare with the Metroplis-Hastings Robbins-Monro (MH-RM) algorithm and an application to the study of the IPIP-NEO personality inventory data is given to assess the performance of the new algorithm. Extensions of the proposed algorithm for application on more complicated models and different data types are also discussed.

On the Structure of Quantum Markov Chains on Cayley Trees Associated with Open Quantum Random Walks

Quantum Markov chains (QMCs) and open quantum random walks (OQRWs) represent different quantum extensions of the classical Markov chain framework. QMCs stand as a more profound layer within the realm of Markovian dynamics. The exploration of both QMCs and OQRWs has been a predominant focus over the past decade. Recently, a significant connection between QMCs and OQRWs has been forged, yielding valuable applications. This bridge is particularly impactful when studying QMCs on tree structures, where it intersects with the realm of phase transitions in models naturally arising from quantum statistical mechanics. Furthermore, it aids in elucidating statistical properties, such as recurrence and clustering. The objective of this paper centers around delving into the intricate structure of QMCs on Cayley trees in relation to OQRWs. The foundational elements of this class of QMCs are built upon using classical probability measures that encompass the hierarchical nature of Cayley trees. This exploration unveils the pivotal role that the dynamics of OQRWs play in shaping the behavior of the Markov chains under consideration. Moreover, the analysis extends to their classical counterparts. The findings are further underscored by the examination of notable examples, contributing to a comprehensive understanding of the outcomes.

Qualitative reachability for open interval Markov chains.

Interval Markov chains extend classical Markov chains with the possibility to describe transition probabilities using intervals, rather than exact values. While the standard formulation of interval Markov chains features closed intervals, previous work has considered open interval Markov chains, in which the intervals can also be open or half-open. In this article we focus on qualitative reachability problems for open interval Markov chains, which consider whether the optimal (maximum or minimum) probability with which a certain set of states can be reached is equal to 0 or 1. We present polynomial-time algorithms for these problems for both of the standard semantics of interval Markov chains. Our methods do not rely on the closure of open intervals, in contrast to previous approaches for open interval Markov chains, and can address situations in which probability 0 or 1 can be attained not exactly but arbitrarily closely.

Mean-field dynamics of open quantum systems with collective operator-valued rates: validity and application

We consider a class of open quantum many-body Lindblad dynamics characterized by an all-to-all coupling Hamiltonian and by dissipation featuring collective ‘state-dependent’ rates. The latter encodes local incoherent transitions that depend on average properties of the system. This type of open quantum dynamics can be seen as a generalization of classical (mean-field) stochastic Markov dynamics, in which transitions depend on the instantaneous configuration of the system, to the quantum domain. We study the time evolution in the limit of infinitely large systems, and we demonstrate the exactness of the mean-field equations for the dynamics of average operators. We further derive the effective dynamical generator governing the time evolution of (quasi-) local operators. Our results allow for a rigorous and systematic investigation of the impact of quantum effects on paradigmatic classical models, such as quantum generalized Hopfield associative memories or (mean-field) kinetically-constrained models.