Markov Chain Research Articles

Latent Markov time-interaction processes

We present parametric and semiparametric latent Markov time-interaction processes, that are point processes where the occurrence of an event can increase or reduce the probability of future events. We first present time-interaction processes with parametric and nonparametric baselines, then we let model parameters be modulated by a discrete state continuous time latent Markov process. Posterior inference is based on a novel and efficient data augmentation approach in the Markov Chain Monte Carlo framework. We illustrate with a simulation study; and an original application to terrorist attacks in Europe in the period 2001-2017, where we find two distinct latent clusters for the hazard of occurrence of terrorist events, negative association with GDP growth, and self-exciting phenomena.

Research on Complete Coverage Path Planning of Agricultural Robots Based on Markov Chain Improved Genetic Algorithm

Due to the limitations of low coverage, high repetition rate, and slow convergence speed of the basic genetic algorithm (GA) in robot complete coverage path planning, the state transition matrix of the Markov chain is introduced to guide individual mutation based on the genetic mutation path planning algorithm, which can improve the quality of population individuals, enhancing the search ability and convergence speed of the genetic algorithm. The proposed improved genetic algorithm is used for complete coverage path planning simulation analysis in different work areas. The analysis results show that compared to traditional genetic algorithms, the improved genetic algorithm proposed in this paper reduces the average path length by 21.8%, the average number of turns by 6 times, the repetition rate by 83.8%, and the coverage rate by 7.76% in 6 different work areas. The results prove that the proposed improved genetic algorithm is applicable in complete coverage path planning. To verify whether the Markov chain genetic algorithm (MCGA) proposed is suitable for agricultural robot path tracking and operation, it was used to plan the path of an actual land parcel. An automatic navigation robot can track the planned path, which can verify the feasibility of the MCGA proposed.

A multi-phase mission success evaluation approach for maritime autonomous surface ships considering equipment performance degradation and system composition changes

With the development of Maritime Autonomous Surface Ships (MASSs), the mission success assessment problem cannot be ignored because of potential security issues of MASSs. This paper addresses the issue of performance degradation at different stages, the problem of state dependence in multi-stage missions of MASSs, and the correlation problem between missions of stages. Based on the coupling of the reliability model of each single-stage mission system, a multi-stage mission success evaluation method for MASSs is proposed. By leveraging the ability of the conditional probability tables in Bayesian networks to express the complex relationships between the nodes, the dynamic Bayesian network model of stages is constructed based on the fault tree. Based on the Markov process, the problem of state dependence on shared equipment between stages is solved. Considering the complex relationship between multi-stage missions, the virtual node is introduced, and the multi-state Bayesian network is combined to realize the coupling of the reliability evaluation results of each single-stage mission. It is applied to the multi-stage mission success evaluation of the MASS to obtain the success probability of MASSs and the key equipment of each stage. The results show that evaluation results are more suitable for engineering practice.

Heterogeneous Mediation Analysis for Cox Proportional Hazards Model With Multiple Mediators.

This study proposes a heterogeneous mediation analysis for survival data that accommodates multiple mediators and sparsity of the predictors. We introduce a joint modeling approach that links the mediation regression and proportional hazards models through Bayesian additive regression trees with shared typologies. The shared tree component is motivated by the fact that confounders and effect modifiers on the causal pathways linked by different mediators often overlap. A sparsity-inducing prior is incorporated to capture the most relevant confounders and effect modifiers on different causal pathways. The individual-specific interventional direct and indirect effects are derived on the scale of the logarithm of hazards and survival function. A Bayesian approach with an efficient Markov chain Monte Carlo algorithm is developed to estimate the conditional interventional effects through the Monte Carlo implementation of the mediation formula. Simulation studies are conducted to verify the empirical performance of the proposed method. An application to the ACTG175 study further demonstrates the method's utility in causal discovery and heterogeneity quantification.

Within-chain parallelization-Giving Stan Jet Fuel for population modeling in pharmacometrics.

Stan is a powerful probabilistic programming language designed mainly for Bayesian data analysis. Torsten is a collection of Stan functions that handles the events (e.g., dosing events) and solves the ODE systems that are frequently present in pharmacometric models. To perform a Bayesian data analysis, most models in pharmacometrics require Markov Chain Monte Carlo (MCMC) methods to sample from the posterior distribution. However, MCMC is computationally expensive and can be time-consuming, enough so that people will often forgo Bayesian methods for a more traditional approach. This paper shows how to speed up the sampling process in Stan by within-chain parallelization through both multi-threading using Stan's reduce_sum() function and multi-processing using Torsten's group ODE solver. Both methods show substantial reductions in the time necessary to sufficiently sample from the posterior distribution compared with a basic approach with no within-chain parallelization.

A Conceptual Model for Health Impact Assessment of Personalized Prevention Policies

Abstract Background Personalized prevention and pharmacogenomics are emerging fields in global health, promoting tailored approaches to disease prevention and individualized drug therapy, based on the individuals’ genetic profile. However, adoption of these technologies in public health is challenged by the possibility of health disparities. Health Impact Assessment (HIA) is a useful tool to address this issue, by examining how policies differentially impact population groups, guiding stakeholders to more equitable and inclusive outcomes. Solid frameworks for assessing health impacts are needed to avoid heterogeneity when accounting for equity outcomes in HIA. This study tested a conceptual model for HIA in personalized medicine, using a case study in pharmacogenetics to map potential impacts and outcomes, with a focus on health equity. Methods A conceptual model for a HIA was developed to map expected impacts and outcomes of a health policy, aimed at enforcing DPYD genotype-guided dosing to prevent fluoropyrimidine toxicity in colorectal cancer patients. The conceptual model was developed based on literature reviews and consultation with stakeholders. Identification of relevant impacts guided selection of indicators and quantitative impact assessment using a Markov chain model. Results Impact pathways were identified using a three-tiered reference framework encompassing individual and population perspectives, organizational factors, and overarching health system considerations. The final diagram outlined key outcomes in four categories: health, organizational, economic and equity. HIA projections using this model were obtained for the DPYD testing policy. Conclusions In the rise of personalized medicine and pharmacogenomics, health inequalities challenges need to be addressed before implementation in healthcare. This study provided a novel HIA conceptual model for personalized prevention policies based on genomic information, with an emphasis on equity. Key messages • HIA is an adequate tool to evaluate potential health inequalities resulting from personalized medicine policies in healthcare. • This study provides a validated framework for HIA of personalized prevention policies grounded in genomic data.

Variational Bayesian Monte Carlo simulation for rare event assessment

Estimating rare events in structural engineering demands intensive computational resources, and the complexity of the performance function poses challenges to accurate probability estimation. To efficiently conduct the structural reliability analysis on rare events, we propose a novel variational Bayesian Monte Carlo (VBMC)-based subset simulation (SS) method. The proposed approach can be divided into two aspects: the Monte Carlo simulation (MCS) is first applied to estimate the first-layer unconditional failure probability, then VBMC is employed to estimate subsequent conditional failure probabilities. Finally, the original small failure probability is obtained by the product of the first-layer unconditional failure probability and a series of conditional failure probabilities. The proposed approach inherits the merits of SS as well as VBMC, which converts the tricky rare event estimation into a series of high-frequency event estimations and leverages VBMC instead of the canonical Markov chain to produce the independent and informative next-generation failure-conditional samples. Four case studies, including high-dimensional and discrete state space scenarios, are performed to illustrate the feasibility and generality of the proposed method.

A stochastic analysis of the dynamic space utilization in warehouses

Improving the volume space utilization of modern warehouses is a critical objective in warehouse layout design, making the accurate estimation of space utilization across different layouts essential. However, the dynamic demand and replenishment processes of heterogeneous stock keeping units (SKUs) in modern warehouses present significant challenges in accurately assessing space utilization. Traditional models have relied on deterministic frameworks that assume constant demand and production processes, which do not fully capture the heterogeneity and stochasticity in demand arrivals, sizes, and replenishment times encountered in practical scenarios. This study aims to develop a framework for computing volume space utilization in modern block stacking warehouses, incorporating both heterogeneity and stochasticity. We investigate two models in warehouse scenarios: the lost sale model, where excess demand is lost, and the backorder model, where unmet demand is backlogged. For each model, we derive the closed-form expressions of three key types of space wastes that characterize the space utilization of warehouses: honeycombing, aisle, and top-of-lane space wastes, using a continuous-time Markov chain analytical framework. Utilizing these expressions, we analyze the tradeoffs among these wastes and identify the optimal lane depth that maximizes space utilization. Our case studies show that, in a stochastic environment, the proposed computational framework allows for a more accurate estimation of space utilization, and the application of the obtained optimal lane depth can achieve significantly higher space utilization than deterministic methods found in the literature. This underscores the importance of incorporating stochasticity and heterogeneity of demand and replenishment in layout design. Additionally, by investigating the sensitivity of space utilization to varying demand processes and replenishment strategies, we provide managerial insights for adapting warehouse layout designs in response to changes in SKUs' demand and replenishment patterns.

QPOs from charged particles around magnetized black holes in braneworlds

Quasiperiodic oscillations (QPOs) are a powerful tool for testing gravity theories, probing gravitational and electromagnetic field properties, and obtaining constraints on the black hole and field parameters. This work considers charged particle dynamics near uniformly magnetized black holes in braneworlds. First, we obtain the solution of the Maxwell equation for magnetic fields and calculate the radial and angular magnetic field components. We derive and analyze the effective potential of charged particles for circular orbits and investigate the energy and angular momentum for the circular orbits. We also analyze the combined effects of magnetic interaction and braneworlds on the charged particles’ innermost stable circular orbits (ISCOs). We calculate the angular momentum of charged particles in Keplerian orbits in the presence of an external magnetic field and braneworlds. Also, we investigate frequencies of the particle oscillations along vertical and angular directions. We applied our studies on particle oscillations to the QPO studies in the relativistic precession model. Finally, we obtain constraints on magnetic interaction and braneworld parameters together with the black hole mass and QPO orbits using Monte Carlo Markov Chain (MCMC) simulation in the four-dimensional parameter space for the QPOs observed in the microquasars XTE J1550-564, GRO J1655-40 & GRS 1915-105, and at the center of galaxies M82 and Milky Way.

Reference to Patients in Nurse Shift Handover Meetings: Exploring the Dynamics of Referring Expressions

ABSTRACT This paper investigates the dynamics of referring expressions in hospital nurse handover meetings when discussing patients. We apply the methods of the Variable Length Markov Chain (VLMC) and network analyses to model the use of referring expressions and evaluate relationships between them. The models reveal second-order dependencies emerging for metonymy and noun phrases. Specifically, metonymy shows a greater association with the beginning of a reference, particularly in the context of other metonymies. In contrast, noun phrases tend to be more strongly associated with later points in the reference. Further, we introduce the notion of referential typicality, which measures the conformity of sequences of referring expressions to anticipated patterns. We show, for example, that consecutive noun phrases fall outside the typical pattern, whereas metonymical sequences and sequences of pronouns are highly typical. The transitions from metonymy or nouns to pronouns also closely align with a highly typical pattern. Using a Generalized Additive Model (GAM), we then track the overall evolution of referential typicality throughout the duration of handover meetings, from their beginning to end. The study reveals a subtle increase in referential typicality towards the end of these sessions, indicating a trend towards more consistent referencing as the discourse unfolds.

Langevin dynamics for the probability of finite state Markov processes

Langevin dynamics for the probability of finite state Markov processes

Efficient sampling using macrocanonical Monte Carlo and density of states mapping

In the context of Monte Carlo sampling for lattice models, the complexity of the energy landscape often leads to Markov chains being trapped in local optima, thereby increasing the correlation between samples and reducing sampling efficiency. This study proposes a Monte Carlo algorithm that effectively addresses the irregularities of the energy landscape through the introduction of the estimated density of states. This algorithm enhances the accuracy in the study of phase transitions and is not model specific. Although our algorithm is primarily demonstrated on the two-dimensional square lattice model, the method is also applicable to a broader range of lattice and higher-dimensional models. Furthermore, the study develops a method for estimating the density of states of large systems based on that of smaller systems, enabling high-precision density of states estimation within specific energy intervals in large systems without additional sampling. For regions of lower precision, a reweighting strategy is employed to adjust the density of states to enhance the precision further. This algorithm is not only significant within the field of lattice model sampling but may also inspire applications of the Monte Carlo method in other domains. Published by the American Physical Society 2024

Partition function approach to non-Gaussian likelihoods: macrocanonical partitions and replicating Markov-chains

Monte-Carlo techniques are standard numerical tools for exploring non-Gaussian and multivariate likelihoods. Many variants of the original Metropolis-Hastings algorithm have been proposed to increase the sampling efficiency. Motivated by Ensemble Monte Carlo we allow the number of Markov chains to vary by exchanging particles with a reservoir, controlled by a parameter analogous to a chemical potential μ, which effectively establishes a random process that samples microstates from a macrocanonical instead of a canonical ensemble. In this paper, we develop the theory of macrocanonical sampling for statistical inference on the basis of Bayesian macrocanonical partition functions, thereby bringing to light the relations between information-theoretical quantities and thermodynamic properties. Furthermore, we propose an algorithm for macrocanonical sampling, 𝙰𝚟𝚊𝚕𝚊𝚗𝚌𝚑𝚎 𝚂𝚊𝚖𝚙𝚕𝚒𝚗𝚐, and apply it to various toy problems as well as the likelihood on the cosmological parameters Ωm and w on the basis of data from the supernova distance redshift relation.

Flow-based nonperturbative simulation of first-order phase transitions

We present a flow-based method for simulating and calculating nucleation rates of first-order phase transitions in scalar field theory on a lattice. Motivated by recent advancements in machine learning tools, particularly normalizing flows for lattice field theory, we propose the “partitioning flow-based Markov chain Monte Carlo (PFMCMC) sampling” method to address two challenges encountered in normalizing flow applications for lattice field theory: the “mode-collapse” and “rare-event sampling” problems. Using a (2+1)-dimensional real scalar model as an example, we demonstrate the effectiveness of our PFMCMC method in modeling highly hierarchical order parameter probability distributions and simulating critical bubble configurations. These simulations are then used to facilitate the calculation of nucleation rates. We anticipate the application of this method to (3+1)-dimensional theories for studying realistic cosmological phase transitions.

Evaluating MCMC Convergence in a Bayesian Model of ST-Elevation Myocardial Infarction Female Patients in Malaysia

The majority of research that looked at the Bayesian Markov Chain Monte Carlo (MCMC) approach for prognostic modelling of cardiovascular disease only focused on the use of the Bayesian approach in variable selection, model selection, and prior distribution selection. But very few of this research has looked at the Markov chains' convergence in the model. In this study, convergence diagnostics were carried out to evaluate the convergence of Markov chains using both visual inspection and additional diagnostics. The National Cardiovascular Disease Database-Acute Coronary Syndrome (NCVD-ACS) registry, which included 1248 female patients with ST-Elevation Myocardial Infarction (STEMI) between 2006 and 2013, was used for this study's analysis. The multivariate Bayesian model identified six significant variables: dyslipidaemia, myocardial infarction, smoking, renal disease, Killip class, and age group. The trace plots did not reveal any distinctive patterns based on these significant variables, and the model's MCMC mixing is typically good. While for the autocorrelation plots, mild autocorrelations for age group, Killip IV, as well as the intercept term in the model. Since there were only mild autocorrelations, no thinning is needed. Also, the Geweke diagnostic showed that the chain is divided into two windows containing a set fraction of the first and last iterations which produced standard Z-scores. The Geweke diagnostic did not provide evidence of non-convergence, as none of the Z-scores fell in the extreme tails of the N (0,1). In this study, a number of plots and additional diagnostic tools showed that the Markov chains have reached convergence, which is relevant to the general use of the MCMC approach.

Real‐time reliability evaluation method for IGBT module in MMC based on junction temperature and bond wire failure online monitoring

AbstractThe lack of an MMC reliability evaluation approach that incorporates data on essential components' health monitoring in the already related research prevents reliability from being further improved. Therefore, this article proposes a real‐time reliability evaluation method of the multi‐state IGBT module in MMC through iterative calculation of the Markov chain model. The degradation process of the IGBT module is divided into multiple states based on the monitored bond wire failure, and the failure rate of the IGBT module is updated in real time using the accumulated monitoring data of junction temperature. The proposed method can more accurately reflect the real state of MMC compared with the binary state evaluation method. Besides, a multi‐channel high‐speed data acquisition board is designed to measuring the gate voltage and collector voltage of the IGBT for the health evaluation of IGBT modules in MMC; the typically measured results validate the effectiveness of the data acquisition board.

Embedding and elimination for performance analysis in stochastic process algebra dtsdPBC

ABSTRACT dtsdPBC extends the well-known algebra of parallel processes, Petri box calculus (PBC), by incorporating discrete time stochastic and deterministic delays. To analyze performance in this extended calculus, the underlying semi-Markov chains, and the related (complete) and reduced discrete time Markov chains of the process expressions are built. The semi-Markov chains are extracted using the embedding method, which constructs the embedded discrete time Markov chains and calculates the sojourn time distributions in the states. The reductions of the discrete time Markov chains are obtained through the elimination method, which removes the vanishing states (those with zero sojourn times) and recalculates the transition probabilities among the tangible states (those with positive sojourn times). We prove that the reduced semi-Markov chain coincides with the reduced discrete time Markov chain, by demonstrating that an additional embedding into the reduced semi-Markov chain is needed for the reduced embedded discrete time Markov chain to match the embedded reduced discrete time Markov chain, and by comparing the respective sojourn times.

Bayesian Empirical Likelihood Regression for Semiparametric Estimation of Optimal Dynamic Treatment Regimes.

We propose a semiparametric approach to Bayesian modeling of dynamic treatment regimes that is built on a Bayesian likelihood-based regression estimation framework. Methods based on this framework exhibit a probabilistic coherence property that leads to accurate estimation of the optimal dynamic treatment regime. Unlike most Bayesian estimation methods, our proposed method avoids strong distributional assumptions for the intermediate and final outcomes by utilizing empirical likelihoods. Our proposed method allows for either linear, or more flexible forms of mean functions for the stagewise outcomes. A variational Bayes approximation is used for computation to avoid common pitfalls associated with Markov Chain Monte Carlo approaches coupled with empirical likelihood. Through simulations and analysis of the STAR*D sequential randomized trial data, our proposed method demonstrates superior accuracy over Q-learning and parametric Bayesian likelihood-based regression estimation, particularly when the parametric assumptions of regression error distributions may be potentially violated.

Sampling random graphs with specified degree sequences

The configuration model is a standard tool for uniformly generating random graphs with a specified degree sequence, and is often used as a null model to evaluate how much of an observed network’s structure can be explained by its degree structure alone. A Markov chain Monte Carlo (MCMC) algorithm, based on a degree-preserving double-edge swap, provides an asymptotic solution to sample from the configuration model. However, accurately and efficiently detecting when this Markov chain is sufficiently close to its stationary distribution remains an unsolved problem. Here, we provide a solution to sample from the configuration model using this standard MCMC algorithm. We develop an algorithm, based on the assortativity of the sampled graphs, for estimating the gap between effectively independent MCMC states, and a computationally efficient gap-estimation heuristic derived from analyzing a corpus of 509 empirical networks. We provide a convergence detection method based on the Dickey-Fuller Generalized Least Squares test, which we show is more accurate and efficient than three alternative Markov chain convergence tests. Supplementary materials for the proposed methods can be found here.

Reaction kinetics for catalytic hydrogenation of quinoline to decahydroquinoline as liquid organic hydrogen carrier

The catalytic hydrogenation of quinoline to decahydroquinoline (DHQ) formation is studied for various parameters such as temperature, hydrogen pressure, reaction time, solvents, reactant-to-solvent ratio, and catalyst loading over 5%Pd loaded on alumina (Pd/Al2O3) to obtain optimal reaction conditions. The hydrogen uptake of quinoline in the liquid phase reaction using isopropylalcohol (IPA) is studied in an autoclave reactor. The optimum reaction parameters of 50 bar H2 pressure and 175oC with reactant to solvent ratio of 1:9 for reaction time of 5 h are observed. The effect of IPA solvent showed that hydrogen uptake of 6.91 wt% with complete hydrogenation of quinoline and DHQ yield of more than 99% is observed. The reaction kinetic model is developed for a simplified reaction mechanism and is simulated using the Markov Chain Monte Carlo (MCMC) simulation tool to predict the rate constants and the experimental observations are validated with the model predictions. The activation energy for quinoline hydrogenation to py-THQ formation is estimated to be 136.57 kJ/mol. It is envisaged that quinoline hydrogenation to pz-THQ is the rate-limiting step in the DHQ formation.