Finite Markov Chain Research Articles

Risk conscious investment

The risk conscious investor is defined as the maximizer of a conservative valuation or a dynamic nonlinear expectation. Both the static and dynamic problems are addressed using distortions of tail probabilities or distortions of tail measures. The multivariate static problem is solved in the context of the multivariate bilateral gamma model. For the dynamic model states and their transitions are modeled in two ways. The first defines states by a parametric model for the return distribution with transitions described by a continuous-time finite state Markov chain between the distributional possibilities. In the second model states are represented by the first four power variations with transitions given by (OU) equations for modeling stochastically the first four power variations as Tempered Fractional Lévy Processes (TFLP). Numerical solutions for policy functions are implemented in trading 768 equity assets over seven years ending December 2021.

Political institutions and output collapses

This paper examines whether major output collapses are more likely under autocracy. Using data on 123 developing countries over 1971–2016, we model the joint evolution of output growth and political institutions as a finite state Markov chain with a two-dimensional state space. We study how countries move between states. We find that growth is more likely to be sustained under democracy than under autocracy; output collapses are more likely to deepen under autocracy; and stagnation under autocracy can give way to outright collapse. Democratic countries appear to be more resilient.

Metastable Γ-expansion of finite state Markov chains level two large deviations rate functions

Metastable Γ-expansion of finite state Markov chains level two large deviations rate functions

Reliability analysis and preventive maintenance policy for consecutive k$k$‐out‐of‐n:F$n: F$ balanced system under failure criterion operating in shock environment

AbstractThis paper presents a consecutive ‐out‐of‐: balance system in a shock environment, where the state transition of component is induced by external shocks. If a predetermined threshold number of effective shocks are applied to component in a critical state, the component will fail. The state of the system is defined by the number of consecutive failing component groups in the system, which leads to system failure when a critical number of consecutive failing components is reached. To minimize maintenance costs, we propose a preventive maintenance method with an optimization model. We use finite Markov chain imbedding and Phase‐type distribution to calculate component group failure rates and associated probability functions in discrete and continuous time, respectively. The validity and accuracy of the model are confirmed by numerical examples and Monte Carlo simulations.

ROLE OF INTERVENTION STRATEGIES AND PSYCHOLOGICAL EFFECT ON THE CONTROL OF INFECTIOUS DISEASES IN THE RANDOM ENVIRONMENT

Infectious disease outbreaks have caused significant losses to populations worldwide in terms of morbidity, social and economic costs. This has prompted governments to seek suitable intervention policies to suppress the outbreaks. This paper aims to explore the intrinsic impact of intervention strategies and the psychological behavior of people on the spread of infectious diseases. Employing the finite state Markov chain, we formulate a new stochastic SIS epidemic model under regime switching by using the Beddington–DeAngelis transmission function, in the presence of some governmental control measures. The study of the model is centered on three different scenarios in terms of varying two control parameters. We analyze the deterministic system for the global dynamics in terms of basic reproductive number. Some new threshold parameters are defined to discuss the extinction and the persistence of the disease in the stochastic setting. Numerical assessments were undertaken to validate theoretical findings for the deterministic and stochastic systems. Our findings strongly suggest that heightened awareness among susceptible individuals contributes significantly to the eradication of infection and fosters a stable epidemiological situation. As levels of public awareness increase, particularly within the susceptible population, a discernible trend toward the eradication of the infection emerges, ultimately ensuring sustained stability within the epidemiological landscape.

Reliability analysis of A combination of [formula omitted] and [formula omitted] systems

In the paper, two-element (f,k) and <f,k> combined systems are developed, and two-kind combined reliability systems and some related new concepts such as irreducible structure and structure equivalence are introduced, which can be used not only in real practical situations, but also enrich the reliability contents. We proved that four-class combined reliability systems need to be considered actually, which facilitates the handling of these combined reliability systems. Meanwhile, a coordinate system with two axes, the number of total failed components and the number of maximal consecutive failed components, is introduced in the paper for representing the working and failed areas for the combined reliability systems, which brings out a useful tool for understanding the working and failure mechanisms, and system state transition ways, for the combined reliability systems. To summarize the reliability calculation, a procedure for calculations of the combined reliability systems is presented, which is an efficient guidance for these reliability computations. Finally, two numerical examples are given to illustrate the computation procedure in which the detailed steps, special on using Finite Markov Chain Imbedding Approach (FMCIA), are shown. We believe that the present paper can shed light on researches on combined reliability systems.

Reliability analysis for a generalized sparse connection multi-state consecutive-k-out-of-n linear system

The utilization of binary states is a common practice in reliability analysis. However, in complex systems such as aviation, spaceflight, and watercraft, the prevalence of multi-state systems and units is more pronounced. Consequently, extensive research has been undertaken to explore reliability analysis with a focus on multi-state consecutive-k models across various disciplines. Notably, the concept of sparse connection has received considerable attention due to its relevance in practical applications like wireless communication, cloud computing networks, and oil pipeline systems. This paper aims to propose a generalized multi-state consecutive-k: G system model that incorporates sparse connection. Both the system and the units are allowed to be in one of the (M+1) possible states, wherein the order numbers 0,1,2,…,M indicate the performance measurement of the system or the units, i.e., 0 denotes the worst performance state while M denotes the best performance state. We classify the model into increasing, decreasing and non-monotonic consecutive-k systems corresponding to different mission requirements in practical applications. This paper utilizes the finite Markov chain imbedding (FMCI) method to derive explicit system state distributions for three types of multi-state consecutive-k-out-of-n: G systems. Subsequently, numerical examples are provided to demonstrate the proposed models and the corresponding results.

Sum-of-Squares Proofs of Logarithmic Sobolev Inequalities on Finite Markov Chains

Sum-of-Squares Proofs of Logarithmic Sobolev Inequalities on Finite Markov Chains

A stochastic Gilpin-Ayala nonautonomous competition model driven by mean-reverting OU process with finite Markov chain and Lévy jumps

A stochastic Gilpin-Ayala nonautonomous competition model driven by mean-reverting OU process with finite Markov chain and Lévy jumps

On a Stochastic Approach to Extensions of the Susceptible‐Infected‐Susceptible (SIS) Model Applied to Malaria

This work presents a stochastic model of malaria spread. We first calculated the basic reproduction number R0 of the models (ShIhRhSh‐SvIv) and (ShLhIhRhSh‐SvLvIv) in order to show that the malaria‐free equilibrium is asymptotically stable; then, we used a finite Markov chain model to describe the interactions between the different compartments of the model (SeLeIeReSe‐SaLaIaRaSa‐SvIv). We carried out numerical simulations of our results for two types of transmission zones: a zone with low malaria transmission and an endemic zone. Through these simulations, we first determined the invariant stationary distribution π∗ of the model, and then, we found that the use of the indoor residual spraying (IRS) method by regular application of insecticides is more effective for the elimination of malaria than the use of long‐acting impregnated mosquito nets (LLINs).

Analytical Expressions of the Markov Chain of K-Ras4B Protein within the Catalytic Environment and a New Markov-State Model

The finite Markov chain to which there correspond the qualities of the conformational dynamics of the K-Ras4B proteins in the catalytic reaction is written. The corresponding Markov-Sates models are studied. The properties of the K-Ras4B processes Markov chain allow one to define a new two-state MSM for the analytical description of the final-state transition. The time evolution of the eigenvalue corresponding to the final-state transition in the Galerkin description is written. The tools for the analytical calculations of the relative error are therefore prepared. New analytical formulations of the time evolution of the eigenvalue corresponding to the final-state transition are newly written from the experimental data and form the properties of the lag time in shaping the discretization error. The features of the discretization error are newly studied. A comparison with the experimental data is proposed.

Optimal Mission Abort Decisions for Multi-Component Systems Considering Multiple Abort Criteria

This paper studies the optimal mission abort decisions for safety-critical mission-based systems with multiple components. The considered system operates in a random shock environment and is required to accomplish a mission during a fixed mission period. If the failure risk of the system is very high, the main mission can be aborted to avoid higher failure cost. The main contribution of this study lies in the design and optimization of mission abort policies for multi-component systems with multiple abort criteria. Moreover, multi-level transitions are considered in this study to characterize the different shock-resistance abilities for components in different states. Mission abort decisions are determined based on the number of components in either defective or failed state. The problem is formulated in the framework of the finite Markov chain imbedding method. We use the Monte-Carlo simulation method to derive the mission reliability and system survivability. Numerical studies and sensitivity analysis are presented to validate the obtained result.

Scaling limits of loop-erased Markov chains on resistance spaces via a partial loop-erasing procedure

We introduce partial loop-erasing operators. We show that by applying a refinement sequence of partial loop-erasing operators to a finite Markov chain, we get a process equivalent to the chronological loop-erased Markov chain. As an application, we construct loop-erased random paths on bounded domains of resistance spaces as the weak limit of the loop erasure of the Markov chains on a sequence of finite sets approximating the space, and the limit is independent of the approximating sequences. The random paths we constructed are simple paths almost surely, and they can be viewed as the loop-erasure of the paths of the diffusion process. Finally, we show that the scaling limit of the loop-erased random walks on the Sierpiński carpet graphs exists, and is equivalent to the loop-erased random paths on the Sierpińksi carpet.

Reliability analysis of multi-state balanced systems with standby components switching mechanism

An increasing attention has been paid to the reliability analysis of balanced system. In previous studies, there were three rebalancing strategies for balanced systems: forcing down the unbalanced pair of subsystems, restarting components that have been shut down, and dynamically adjusting the state of the working components. In this paper, a new rebalancing strategy is proposed by switching the standby components for a multi-state balanced system. The system consists of m subsystems, each with one working component and n-1 standby components. The system is balanced when the working components do not fail and the system's balanced degree doesn't exceed a threshold. Two cases of system balance conditions are considered: one is based on component states and the other is based on symmetric component states. If the system is imbalanced or fails, the system needs to adjust the state of the working component by switching switches to let the system operate normally. The switching rules for each case are given separately. In this paper, the finite Markov chain imbedding approach (FMCIA) is used to derive the system reliability. Finally, numerical examples and sensitivity analysis are given.

A mixed singular/switching control problem with terminal cost for modulated diffusion processes

In this paper, we study the regularity of the value function associated with a stochastic control problem involving two controls that act simultaneously on a modulated multidimensional diffusion process. The first is a switching control modelling a random clock. Whenever the random clock rings, the generator matrix is replaced by another, leading to a different dynamic for the finite state Markov chain of the modulated diffusion process. The second is a singular stochastic control that is executed on the process within each regime.

Joint signatures of two or more semi-coherent systems with shared components

In this paper, for two or more semi-coherent systems with shared components, three signature measures–joint signature, joint survival signature and joint cumulative signature–are presented and their relationship to joint reliability and joint unreliability are also discussed. This relationship provides a joint (survival/cumulative) signature computational method which transforms related computation to the computation of joint reliability/unreliability of the systems. Based on that, further discussions on computation of the joint reliability are presented for two different types of consecutive-k systems, namely, linear (n, f, k): F systems and linear <n, f, k>: F systems, by using finite Markov chain imbedding approach (FMCIA) in a new and novel way. Besides, for modular systems that can be regarded as a series/parallel connection of two groups of subsystems with shared components, their joint (survival/cumulative) signature is derived based on the joint survival/cumulative signatures of the subsystems, and this provides another computational method for joint signatures. Related results are applied to calculate joint signatures of a series/parallel connection of (n, f, k): F subsystems and <n, f, k>: F subsystems by applying the two computational methods together. Finally, some potential applications and generalizations are mentioned.

Reliability analyses of linear two-dimensional consecutive k-type systems

AbstractIn this paper, several linear two-dimensional consecutive k-type systems are studied, which include the linear connected-(k, r)-out-of- $(m,n)\colon\! F$ system and the linear l-connected-(k, r)-out-of- $(m,n)\colon\! F$ system without/with overlapping. Reliabilities of these systems are studied via the finite Markov chain imbedding approach (FMCIA) in a novel way. Some numerical examples are provided to illustrate the theoretical results established here and also to demonstrate the efficiency of the developed method. Finally, some possible applications and generalizations of the developed results are pointed out.

A Markov chain approximation of switched Fokker–Planck equations for a model of on–off intermittency in the postural control during quiet standing

The intermittent on–off switching of feedback control is considered as a major mechanism of postural stabilization during human quiet standing, which can be modeled by switched-type hybrid stochastic delay differential equations with unstable subsystems. Dynamics of the model can also be described by the corresponding switched-type Fokker–Planck (FP) equations. Here, we develop a comprehensive numerical recipe to simulate switched-type FP equations in the case that the probability current is conserved at the switching boundary, as is the case for noise-free models exhibiting C0-continuity for solutions at the boundary, but in a way extendable to cases with discontinuous jumps. Specifically, the FP equations are approximated by a finite state Markov chain model using the finite element method. Then, dynamics of the Markov chain model, including time evolution of probability density function (PDF), stationary PDF, and power spectrum of postural sway are analyzed. We further investigate how the stationary PDF alters as values of important parameters of the model change. Dynamics of the Markov chain model are compared with Monte Carlo-based dynamics of the model, by which the developed numerical recipe is validated. The obtained Markov chain model forms a basis of our future investigations of the intermittent postural control as a Markov decision process.

British Put Option On Stocks Under Regime-Switching Model

In a plain vanilla option, its holder is given the right, but not the obligation, to buy or sell the underlying stock at a specified price (strike price) at a predetermined date. If the exercise date is at maturity, the option is called a European; if the option is exercised anytime prior to maturity, it is called an American. In a British option, the holder can enjoy the early exercise feature of American option whereupon his payoff is the ‘best prediction’ of the European payoff given all the information up to exercise date under the hypothesis that the true drift of the stock equals a specified contract drift. In this paper, in contrast to the constant interest rate and constant volatility assumptions, we consider the British option by assuming that the economic state of the world is described by a finite state continuous-time Markov chain. Also, we provide a solution to a free boundary problem by using PDE arguments. However, closed form expression for the arbitrage-free price are not available in our setting.

A Sequential Test of Traffic Intensity for the M/M/1 Queueing System

This paper deals with testing traffic intensity, which is the most important parameter of the queue system. The Wald-type sequential probability ratio test (SPRT) is performed to determine traffic intensity by taking advantage of the fact that the total number of customers arriving up to the kth service period, including the kth service period, has a negative binomial distribution. Acceptance and rejection limits obtained from the SPRT are arranged as a multiple sampling plan. This plan is applied with a finite Markov chain. Then, the Operating Characteristic (OC) function and the Average Sample Number (ASN) are obtained precisely. Obtaining the standard errors of the sample number compared to the studies on SPRT applications is the novelty brought by this paper. The sequential analysis method based on the finite Markov chain is also applied to a numerical sample.