Discrete Markov Chain Research Articles

Closed-Form Formula for the Conditional Moment-Generating Function Under a Regime-Switching, Nonlinear Drift CEV Process, with Applications to Option Pricing

An analytical derivation of the conditional moment-generating function (MGF) for a regime-switching nonlinear drift constant elasticity of variance process is established. The proposed model incorporates both regime-switching mechanisms and nonlinear drift components to better capture market phenomena such as volatility smiles and leverage effects. Regime-switching models can match the tendency of financial markets to often change their behavior abruptly and the phenomenon that the new behavior of financial variables often persists for several periods after such a change. Closed-form formulas for the MGF under various conditions, which are then applied for option pricing, are also derived. The efficacy and accuracy of the results are validated through a discrete Markov chain simulation. The results obtained from the proposed formulas completely match with those from MC simulations, while requiring significantly less computational time.

Future Changes in Day-to-Day Precipitation Variability in Europe

Abstract We analyzed regional patterns of day-to-day precipitation variability across Europe and assessed their future changes using Coordinated Regional Climate Downscaling Experiment (CORDEX) regional climate models. A discrete Markov chain process was employed to calculate transition probabilities from wet and dry states, and the precipitation variability was quantified using the proposed variability index (IVAR, the sum of wet-to-dry and dry-to-wet transitions divided by the total number of transitions). The IVAR is, in general, lowest in southern Europe and gradually increases northward in the observed data. Performance of the regional climate models is season dependent: They capture IVAR reasonably well in summer, but higher simulated variability was found for the winter season. The IVAR trends computed for the 2006–95 period suggest decreasing day-to-day precipitation variability over southern Europe, especially in summer under the high-concentration RCP8.5 pathway. By contrast, increased variability is projected in northern Europe. Between these two regions, future IVAR trends are less clear because they strongly depend on the selection of driving global model, hinting of an uncertain future hydroclimate in the central European region. Significance Statement In a warming world, water availability will play a key role in ecosystem productivity. Although future changes in rainfall amounts have been studied extensively, much less attention has been given to changes in their temporal distribution and variability. Because grouping wet or dry days into sequences vitally contributes to characterizing floods or droughts, we aimed to study future changes in these tendencies. We found that although future changes in wet or dry days grouping tendencies are mostly driven solely by change in their frequency, climate models do not agree on the change in the frequency of wet days over large parts of continental Europe. This leaves major uncertainties in a future European hydroclimate and implications for impact modeling.

Numerical simulation of neural activity based on discrete Markov chains with stochastic differential equations

Abstract With the development of new numerical calculation methods and computer software science and technology, people can have a good understanding of the potential mechanisms of cerebrovascular diseases. Here, we combine the stochastic differential equation (SDE) of discrete Markov chains to numerically simulate the dynamic changes of neural signals, and find that the changes of neural signals exhibit regular fluctuations. By analyzing the variation of voltage over time, we know that the voltage change at the next moment is closely related to the previous moment and has continuity. Based on the knowledge of neural ion channel dynamics, it was found that there will be longer peak changes in voltage, exhibiting a power-law distribution, which is consistent with the actual situation and statistical data related to resignation channels. By analyzing the voltage and peak changes of ion channels, we can gain a new understanding of the transmission laws of neural information and greatly improve the biological mechanisms.

Information flow and catalytic dyad in SARS-CoV2 main protease Mpro enzyme using embedded discrete Markov chains and centrality measures

Abstract In this work, we use a network representation of the globular crystalline structure of a given protein as a graph structure with N nodes and E edges in order to analyze quantitatively the flow information and to identify key sites within the globular structure. Each node nj represents a $ C_{\alpha}^{i}$ carbon of the main backbone whereas the node’s degree ki is a measure of its physical interactions. In order to identify biologically relevant and active nodes, we compute local per residue closeness $ C_{c}(i)$, betweenness $ C_{b}(i)$ and eigenvector centralities $ C_{e}(i)$. Further analysis is done by embedding a stochastic dynamic discrete Markov chain in order to evaluate the dynamics of a set of normal random walkers (NRW’s) within the network. From this, we compute the mean first passage time matrix M and the stationary occupation probability vector ψi for each node. These two measures provide very useful information on the dynamical process embed within the $ C_{\alpha}$ network. We apply this to SARS-CoV2 Mpro main protease which is a key enzyme in the virus replication cycle. In particular, we focus our attention to the properties of the catalytic dyad integrated by His41-Cys145 in Mpro main protease since this active site has been under intense scrutiny as a pharmaceutical target. In addition, our results show the existence of additional relevant aminoacids that might play a fundamental role on signal propagation and allosteric pathways in SARS-CoV main proteases.

Stochastic occupancy modeling for spaces with irregular occupancy patterns using adaptive B-Spline-based inhomogeneous Markov Chains

This paper presents a discrete time, discrete state-space in-homogeneous Markov Chains model for stochastic occupancy modeling in spaces with irregular occupancy patterns. The goal of the model is to provide accurate predictions of occupancy numbers, enabling appropriate actions to be taken for HVAC system to maintain optimal indoor environment. The proposed Markov Chain model incorporates time in-homogeneity by coupling the time-varying model parameters using a Periodic B-Spline expansion with adaptive knots, which effectively captures patterns in occupancy activity. This method optimizes the distribution of knots based on specific occupancy characteristics observed in different types of rooms. To evaluate the effectiveness of the proposed method, six months of occupancy data collected from a meeting room are utilized. A comprehensive comparison is conducted between the proposed adaptive B-Spline method and other approaches, including the counting method and uniform B-Spline method. The comparison considers both model accuracy and complexity, using metrics such as the Akaike Information Criterion and Bayesian Information Criterion. Results indicate that the proposed model achieves more accurate predictions with fewer model parameters compared to other methods. These forecasts are particularly useful in optimizing the control of HVAC systems, where accurate predictions of future occupancy numbers are essential.

Asymptotic Theorems for Discrete Markov Chains

.

Assessing electric vehicles behavior in power networks: A non-stationary discrete Markov chain approach

Since the inception of the idea to utilize electrically driven vehicles on the power grid, numerous valuable investigations have been carried out to showcase the advantageous capabilities of such technologies. However, there are still uncertainties surrounding the integration of electric vehicles into the power grid. These uncertainties encompass the number of electric vehicles that will be linked to the grid at any given time, the quantity of energy stored in their batteries during both daytime and nighttime, and the impact of their charging patterns on the overall power grid load. Moreover, there are numerous other unanswered queries that demand attention. This study presents a unique model that effectively addresses these uncertainties by utilizing a non-stationary Markov chain. The utilization of a non-stationary discrete Markov model in this study provides a precise and valuable understanding of the constantly evolving and time-dependent nature of electric vehicle behavior in the power network. Through a comprehensive case study, the outputs of the model offer intriguing insights into the number of electric vehicles connected to the grid and the energy they reserve over a 24-hour period. Furthermore, this study assesses the model's accuracy in representing the load modeling of electric vehicle charging.

АСИМПТОТИЧНІ ВЛАСТИВОСТІ ВИПАДКОВИХ МАТРИЦЬ

The work explores the asymptotic properties of large-dimensional stochastic matrices N under the condition of independence of matrix elements or rows (columns). An analysis of the main properties of eigenvalues of stochastic matrices is conducted. The work is dedicated to investigating the asymptotic characteristics of random matrices under the absence of the second moment and also considers the presence of "heavy tails" in the corresponding transitions in the adjacency matrices of the respective graph. The main result of the work is formulated in terms of the transition matrix of a discrete Markov chain and its eigenvalues. In proving the theorem, a non-degenerate Markov chain is considered, describing a mathematical model of random processes that do not change over time with known transition probabilities between states and a corresponding stochastic matrix P, one of whose eigenvalues is unity, and all eigenvalues of the stochastic matrix do not exceed it in absolute value. The proof of this fact follows from the Perron–Frobenius theorem, which concerns the properties of positive matrices and their eigenvalues. Thus, the theorem considered manages to expand the class of random matrices A for which convergence of eigenvalues of the matrix can be applied under the conditions imposed on the elements of the adjacency matrix. Moreover, the imposed conditions are relaxed compared to classical results, where the existence of a finite second moment for the elements of the adjacency matrix is required. This result generalizes both classical results for the normal distribution and similar results of other authors. The obtained result can be used in graph clustering problems to choose the optimal number of clusters, namely, they can be applied to determine the optimal number of clusters in a Grid system, complex networks, in investigations of the structure of the web space, etc.

Retracted: Method of Cumulative Anomaly Identification for Security Database Based on Discrete Markov chain

Retracted: Method of Cumulative Anomaly Identification for Security Database Based on Discrete Markov chain

A Novel Method Based on a Non-Stationary Discrete Markov Chain for Tracking Variations in the Quantity of Reserved Energy and the Number of Electric Vehicles

A Novel Method Based on a Non-Stationary Discrete Markov Chain for Tracking Variations in the Quantity of Reserved Energy and the Number of Electric Vehicles

Hypothesis selection with Monte Carlo tree search for feature-based simultaneous localization and mapping in non-static environments

A static world assumption is often used when considering the simultaneous localization and mapping (SLAM) problem. In reality, especially when long-term autonomy is the objective, this is not a valid assumption. This paper studies a scenario where landmarks can occupy multiple discrete positions at different points in time, where each possible position is added to a multi-hypothesis map representation. A selector-mixture distribution is introduced and used in the observation model. Each landmark position hypothesis is associated with one component in the mixture. The landmark movements are modeled by a discrete Markov chain and the Monte Carlo tree search algorithm is suggested to be used as component selector. The non-static environment model is further incorporated into the factor graph formulation of the SLAM problem and is solved by iterating between estimating discrete variables with a component selector and optimizing continuous variables with an efficient state-of-the-art nonlinear least squares SLAM solver. The proposed non-static SLAM system is validated in numerical simulation and with a publicly available dataset by showing that a non-static environment can successfully be navigated.

Discrete Modeling Approach for Cluster-Based Excess Gibbs-Energy of Molecular Liquids.

The excess Gibbs-energy of a two-component liquid molecular mixture is modeled based on discrete clusters of molecules. These clusters preserve the three-dimensional geometric information about local molecule neighborhoods that inform the interaction energies of the clusters. In terms of a discrete Markov-chain, the clusters are used to hypothetically construct the mixture using sequential insertion steps. Each insertion step and, therefore, cluster is assigned a probability of occurring in an equilibrium system that is determined via the constrained minimization of the Helmholtz free energy. For this, informational Shannon entropy based on these probabilities is used synonymously with thermodynamic entropy. A first approach for coupling the model to real molecules is introduced in the form of a molecular sampling algorithm, which utilizes a force-field approach to determine the energetic interactions within a cluster. An exemplary application to four mixtures shows promising results regarding the description of a variety of excess Gibbs-energy curves, including the ability to distinguish between structural isomers.

A discrete state Markov chain model and decomposition solution method for the transient behaviour of buffered long serial transfer lines

As demand for greater manufacturing efficiency over shorter production runs in serial manufacturing lines increase, models for transient behaviour are required for insight into short-term production operation and control. This paper proposes a method to model the discrete time transient behaviour of paced long serial transfer lines with finite buffers. The method provides for the solution of lines comprised of machines that represent a variety of complex discrete state behaviours. The solution decouples the line into two-machine, one buffer models and adds external starvation and blockage coupling states and harmonizes the system behaviour by solving for transition probabilities into these external states with optimization. The efficacy of the approach is illustrated with three examples, a twenty-machine serial line with geometric machine behaviour, a twenty-machine serial line with complex machine behaviour, and the design of a transient line for a CONWIP strategy.

Analysing ill-conditioned Markov chains.

Discrete state Markov chains in discrete or continuous time are widely used to model phenomena in the social, physical and life sciences. In many cases, the model can feature a large state space, with extreme differences between the fastest and slowest transition timescales. Analysis of such ill-conditioned models is often intractable with finite precision linear algebra techniques. In this contribution, we propose a solution to this problem, namely partial graph transformation, to iteratively eliminate and renormalize states, producing a low-rank Markov chain from an ill-conditioned initial model. We show that the error induced by this procedure can be minimized by retaining both the renormalized nodes that represent metastable superbasins, and those through which reactive pathways concentrate, i.e. the dividing surface in the discrete state space. This procedure typically returns a much lower rank model, where trajectories can be efficiently generated with kinetic path sampling. We apply this approach to an ill-conditioned Markov chain for a model multi-community system, measuring the accuracy by direct comparison with trajectories and transition statistics. This article is part of a discussion meeting issue 'Supercomputing simulations of advanced materials'.

Mathematical Analysis of the Effectiveness of Screening for Intracranial Aneurysms in First-Degree Relatives of Persons with Subarachnoid Hemorrhage

Population screening for aneurysms in patients with risk factors and preventive surgical treatment are beneficial according to numerous studies. One of the most significant risk factors is heredity, namely, the presence of first-degree relatives (FDR) with aneurysmal subarachnoid hemorrhage (aSAH). Nevertheless, there are still no generally accepted approaches or evidence bases regarding the benefits of the aneurysm screening strategy. Mathematical modeling of the dynamics of aneurysm development in the population was carried out using an algorithm implementing a discrete Markov's chain. To implement the model, all probabilities of events and distributions are taken from available literature sources. Three-dimensional time of flight noncontrast magnetic resonance angiography was chosen as a screening method. Patients underwent preventive surgical treatment if an aneurysm was detected. Screening and preventive treatment in the general population reduces the prevalence of aneurysms by 1.74% (3.44% in the FDR group) and the prevalence of aSAH by 14.36% (37.48% in the FDR group). Mortality due to aSAH was reduced by 14.44%. The number of disabilities also decreases. The occurrence of deep disability was reduced by 20.2% in the FDR group. Economic analysis of the part of the population consisting of FDRs showed annual savings of ies also decr CONCLUSIONS: The mathematical model demonstrated that screening and preventive treatment of cerebral aneurysms can reduce aSAH-associated morbidity and mortality. In the FDR group, there was decrease in the prevalence of aSAH and decrease in associated mortality. Screening for cerebral aneurysms is cost-effective.

Transition Path Theory for Langevin Dynamics on Manifolds: Optimal Control and Data-Driven Solver

We present a data-driven point of view for rare events, which represent conformational transitions in biochemical reactions modeled by overdamped Langevin dynamics on manifolds in high dimensions. We first reinterpret the transition state theory and the transition path theory from the optimal control viewpoint. Given a point cloud probing the manifold, we construct a discrete Markov chain with a -matrix computed from an approximated Voronoi tesselation via the point cloud. We use this -matrix to compute a discrete committor function whose level set automatically orders the point cloud. Then based on the committor function, an optimally controlled random walk on point clouds is constructed and utilized to efficiently sample transition paths, which become an almost sure event in time instead of a rare event in the original reaction dynamics. To compute the mean transition path efficiently, a local averaging algorithm based on the optimally controlled random walk is developed, which adapts the finite temperature string method to the controlled Monte Carlo samples. Numerical examples on sphere/torus including a conformational transition for the alanine dipeptide in vacuum are conducted to illustrate the data-driven solver for the transition path theory on point clouds. The mean transition path obtained via the controlled Monte Carlo simulations highly coincides with the computed dominant transition path in the transition path theory.

Scaling limits of bisexual Galton–Watson processes

Bisexual Galton–Watson processes are discrete Markov chains where reproduction events are due to mating of males and females. Owing to this interaction, the standard branching property of Galton–Watson processes is lost. We prove tightness for conveniently rescaled bisexual Galton–Watson processes, based on recent techniques developed in [V. Bansaye, M.E. Caballero, and S. Méléard, Scaling limits of population and evolution processes in random environment, Electron. J. Probab. 24(19) (2019), pp. 1–38]. We also identify the possible limits of these rescaled processes as solutions of a stochastic system, coupling two equations through singular coefficients in Poisson terms added to square roots as coefficients of Brownian motions. Under some additional integrability assumptions, pathwise uniqueness of this limiting system of stochastic differential equations and convergence of the rescaled processes are obtained. Two examples corresponding to mutual fidelity are considered.

Geo/D/1/1 모형에서의 실시간 원격 추정값의 오차 분석

In this paper, we study real-time error in the context of monitoring a binary information source through a delay system. To derive the average real-time error, we model the delay system as a discrete time Geo/D/1/1 queueing model. Using a discrete time three-dimensional Markov chain with finite state space, we analyze the queueing model. We also perform some numerical analysis on various system parameters: state transition probabilities of binary information source; transmission times; and transmission frequencies. When the state changes of the information source are positively correlated and negatively correlated, we investigate the relationship between transmission time and transmission frequency.

An anomaly detection of learning behaviour data based on discrete Markov chain

An anomaly detection of learning behaviour data based on discrete Markov chain

ДИНАМІЧНИЙ МЕТОД ІМІТАЦІЙНОГО МОДЕЛЮВАННЯ ВИКОРИСТАННЯ БОЙОВИХ РЕСУРСІВ ДЛЯ ПРОТИДІЇ ВИСАДЦІ МОРСЬКОГО ДЕСАНТУ НА ОСНОВІ МАРКІВСЬКИХ ЛАНЦЮГІВ

A method and a simulation model for calculating the volume of shells and used artillery pieces during the time necessary to stop the landing of amphibious assault forces is proposed. The simulation model contains. The law of distribution of projectiles. Model of hitting a rectangle of a given shape. The model of the probability of projectiles hitting the target is not less than a given number. Model of movement of types of ships without changing the course. The dynamic method for modelling the use of combat resources to counter an amphibious assault is based on discrete Markov chains. The operational-tactical formulation of the problem contains information that allows using the Markov chain. The definition of the considered system is formulated. The possible states of the system are determined. The step of the process and the time points of the steps are found. Possible transitions of the system from state to state in one step are developed. The conditions for the initial state of the system are accepted. In the problem that has been solved, the system is a ship of the same type with its own properties, which has the goal of reaching the coastline. Conditions accepted. A ship that receives a certain number of hits in the future is not considered. Artillery guns with their own properties that fire at enemy ships in order to prevent ships from reaching the coastline have no restrictions on the number of shells. The conditions for using the model and method are determined, the amount of combat resources to stop the landing is obtained. To increase the likelihood of stopping the landing operation, it is assumed. On the one hand, it is necessary to increase the rate of fire of artillery pieces, significantly improve the quality of firing, which will increase the probability of hitting in conditions of a limited number of shells. On the other hand, an increase in the number of artillery pieces allows you to start destroying enemy ships at shorter distances, which will increase the chance of hitting and reduce the amount of shells used.