Transition Probabilities Of Markov Chain Research Articles

Perpindahan Minat Mahasiswa Terhadap Penggunaan Aplikasi E-Wallet Sebelum dan Setelah Perkuliahan Luring di Masa Pandemi Covid

Technological advances in the field of financial innovation, namely e-wallets, provide convenience and practicality in making transactions online. The use of e-wallets is popular among students, one of which is the Faculty of Science and Technology, Airlangga University. Students with interests or tastes in the type of e-wallet and the running of all activities offline make it very likely that there will be a change in student interest from one e-wallet to another e-wallet. The change in interest will be analyzed with the Markov chain transition probability. The data in this study is primary data by conducting an online survey of 100 respondents in the form of a g-form given to students of the Faculty of Science and Technology, Airlangga University. Based on the results of the Markov chain transition opportunities, it can be concluded that the use of e-wallet applications can move according to the advantages of e-wallet applications in each region. This change in interest in e-wallet application users occurs because the application of choice is an application that provides easy access, lots of discounts, small admin fees, and others.

Non-Fragile H∞ Synchronization for Markov Jump Singularly Perturbed Coupled Neural Networks Subject to Double-Layer Switching Regulation.

This work explores the H∞ synchronization issue for singularly perturbed coupled neural networks (SPCNNs) affected by both nonlinear constraints and gain uncertainties, in which a novel double-layer switching regulation containing Markov chain and persistent dwell-time switching regulation (PDTSR) is used. The first layer of switching regulation is the Markov chain to characterize the switching stochastic properties of the systems suffering from random component failures and sudden environmental disturbances. Meanwhile, PDTSR, as the second-layer switching regulation, is used to depict the variations in the transition probability of the aforementioned Markov chain. For systems under double-layer switching regulation, the purpose of the addressed issue is to design a mode-dependent synchronization controller for the network with the desired controller gains calculated by solving convex optimization problems. As such, new sufficient conditions are established to ensure that the synchronization error systems are mean-square exponentially stable with a specified level of the H∞ performance. Eventually, the solvability and validity of the proposed control scheme are illustrated through a numerical simulation.

Markov Chain transition probability modeling of radon gas records and future projection possibility determination

Markov Chain models serve two purposes firstly, subdivide the main data based on the mean value and one and two standard deviation plus and minus values around the mean value. The second stage of modeling, after calculating the transition probabilities between these categories from the available data, is completed by repeatedly multiplying the categorization date groups until the steady-state transition probability values are obtained. This procedure provides a convenient modeling approach for radon gas transient measurement records. After a brief presentation of Markov Chain procedure in this paper, the application is carried out by considering five categories that lead to a better transition probability matrix. Such a matrix provides information about the probabilities of future transition at the radon data measuring station. In addition, it is possible to associate these transition probabilities with the possibility of an earthquake.

MULTI-STATE MODEL FOR CALCULATION OF LONG-TERM CARE INSURANCE PRODUCT PREMIUM IN INDONESIA

Long Term Care (LTC) insurance is a type of health insurance. One of the LTC products is Annuity as A Rider Benefit. This insurance provides benefits for medical care costs during the term and death benefits if the insured dies. This insurance product can be modeled with a multi-state model. The multi-state model is a stochastic process in which the subject can switch states at a specified number of states. This paper discusses the calculation of LTC insurance premiums with the Annuity as A Rider Benefit product using a multi-state model for critically ill patients in Indonesia. The state used consisted of eight states, namely healthy, cancer, heart disease, stroke, died from the illness from each disease, and died from others. The premium calculation also utilized Markov chain transition probabilities. The data used were data on Indonesia's population in 2018, data on the prevalence of cancer, heart disease, stroke, and Indonesia's 2019 mortality table. The stages of this study were calculating the net single premium value, benefit annuity value, and insurance premium value. The case study was conducted on a 25 years old male in good health following LTC insurance with a coverage period of 5 years. It was known that the compensation value for someone who dies was IDR 100,000,000 and the interest rate used was 5%. The calculation results obtained an annual premium of IDR 5,308,915 which was then varied based on gender and varied interest. Insurance premiums for men were more expensive than for women since men had a greater chance of dying. Then, the higher the interest rate taken; the lower premium paid. This was because the interest rate is a discount variable.

A greedy chip‐firing game

AbstractWe introduce a deterministic analogue of Markov chains that we call the hunger game. Like rotor‐routing, the hunger game deterministically mimics the behavior of both recurrent Markov chains and absorbing Markov chains. In the case of recurrent Markov chains with finitely many states, hunger game simulation concentrates around the stationary distribution with discrepancy falling off like , where is the number of simulation steps; in the case of absorbing Markov chains with finitely many states, hunger game simulation also exhibits concentration for hitting measures and expected hitting times with discrepancy falling off like rather than . When transition probabilities in a finite Markov chain are rational, the game is eventually periodic; the period seems to be the same for all initial configurations and the basin of attraction appears to tile the configuration space (the set of hunger vectors) by translation, but we have not proved this.

Time-varying gain controller synthesis of piecewise homogeneous semi-Markov jump linear systems

This paper is devoted to the time-varying gain controller synthesis problem for a class of discrete-time piecewise homogeneous semi-Markov jump linear systems (SMJLSs). Both sojourn-time probability mass functions and embedded Markov chain transition probability are simultaneously regulated by high-level Markov chain, which leads to a more general piecewise homogeneous semi-Markov kernel. A novel class of multivariate-dependent Lyapunov function is constructed, which is mode-dependent, elapsed-time-dependent, and variation-dependent. The objective of this paper is to design two classes of time-varying gain controllers, i.e., elapsed-time-dependent controller and finite memory controller under bound sojourn time. Numerically testable stabilization criteria are established for discrete-time piecewise homogeneous SMJLSs via the proposed novel Lyapunov function and two classes of controllers. Furthermore, two desired stabilizing controllers are designed such that the closed-loop system is mean-square stable. Finally, the proposed control strategies are applied to a numerical example and a practical example of solar thermal receiver to show the effectiveness and applicability of the theoretical results.

Dynamics of Chinese Export Comparative Advantage: Analysis Based on RSCA Index

Based on RSCA index, using kernel density estimation, Markov chain transition probability matrix, and survival analysis method, this paper analyzes the dynamics of Chinese export comparative advantage from 2001 to 2020 and draws the following conclusions. Firstly, after 20 years of export trade development, although the comparative advantage of a few commodities of China has weakened and comparative disadvantage has increased, the comparative advantage of most commodities is improving, and the overall distribution of comparative advantage remained unchanged. Secondly, the stability of Chinese comparative advantage is higher than liquidity, and liquidity as a whole shows a good trend. In addition, the viability of Chinese commodities with comparative advantage has performed well in the past 20 years. Therefore, China should optimize export mode based on comparative advantage.

PNS90 Estimation of Markov Chain Transition Probabilities and Rates for Use As Inputs to Multistate Models

Post-thrombotic syndrome (PTS) is a chronic venous disease that can occur after deep-vein thrombosis and substantially impacts on quality of life. This paper develops and extends Markov Chain Monte Carlo (MCMC) methods (Welton 2005) to estimate the rates of transition between PTS health states using published summary data. A clinical study reported aggregate baseline and follow-up outcomes for 97 PTS patients, of whom 46(47%) were severe and 39(40%) moderate PTS (Black 2018). After 2 years, 7 were severe, 6 moderate, 22 mild and 18 had no PTS, with 44 missing data. A MCMC model was constructed in Winbugs to estimate the underlying rates of transition between severe PTS to moderate (r1), moderate to mild (r2) and mild to no PTS (r3), taking account that transitions are unobserved and assuming missing data were missing completely at random. The set of Kolmogorov forward equations required to map between transition rates and probabilities were calculated algebraically using matrix decomposition (Jones 2017). The MCMC model estimated mean (standard error) rates of r1=0.52(0.23), r2=2.16(0.50), and r3=0.83(0.21). These rates make predictions that fit reasonably well with the observed data at 2 years, predicting with the same baseline patients, 7 severe, 5 moderate, 23 mild and 18 no PTS at 2 years. The MCMC model made estimates that fit well with observed data to be used as baseline rates in a state-transition decision model. The advantage of estimating rates using MCMC rather than other methods for obtaining transition probabilities (e.g. assuming a multivariate Dirichlet distribution) is that rates can be directly multiplied by relative treatment effects obtained from literature review of randomized controlled trials, and permit more straightforward extrapolation beyond the time horizon of the original study.

Cluster consensus of nonlinear multi-agent systems with Markovian switching topologies and communication noises

This paper focuses on the mean square cluster consensus of nonlinear multi-agent systems with Markovian switching topologies and communication noise via pinning control technique. Network topology can take weaker conditions in each cluster but an extra balanced condition is also needed. A time-varying control gain will be introduced to eliminate the effect of stochastic noise. For the case of fixed topology, if the induced digraph of each cluster has a directed spanning tree, the sufficient conditions for the mean square cluster consensus can be obtained. For the case of Markovian switching topologies, if the induced digraph of union of the Laplacian matrix of each mode has a directed spanning tree, the mean square cluster consensus conclusion can be derived. Particularly, if the elements of transition probability of Markov chain are partly unknown, we can also obtain the same conclusion under the same conditions. Finally, two examples are given to illustrate our results.

A Novel Measure Inspired by Lyapunov Exponents for the Characterization of Dynamics in State-Transition Networks

The combination of network sciences, nonlinear dynamics and time series analysis provides novel insights and analogies between the different approaches to complex systems. By combining the considerations behind the Lyapunov exponent of dynamical systems and the average entropy of transition probabilities for Markov chains, we introduce a network measure for characterizing the dynamics on state-transition networks with special focus on differentiating between chaotic and cyclic modes. One important property of this Lyapunov measure consists of its non-monotonous dependence on the cylicity of the dynamics. Motivated by providing proper use cases for studying the new measure, we also lay out a method for mapping time series to state transition networks by phase space coarse graining. Using both discrete time and continuous time dynamical systems the Lyapunov measure extracted from the corresponding state-transition networks exhibits similar behavior to that of the Lyapunov exponent. In addition, it demonstrates a strong sensitivity to boundary crisis suggesting applicability in predicting the collapse of chaos.

Estimates of stability of transition probabilities for non-homogeneous Markov chains in the case of the uniform minorization

Estimates of stability of transition probabilities for non-homogeneous Markov chains in the case of the uniform minorization

АНАЛИЗ И ИССЛЕДОВАНИЕ ПЕРЕНАЛАЖИВАЕМОСТИ ОБОРУДОВАНИЯ ДЛЯ ВНУТРЕННЕЙ МЕЛКОРАЗМЕРНОЙ РЕЗЬБООБРАБОТКИ

The article presents the results of analysis and theoretical research in the direction of improving equipment for internal threading of parts in a flexible automated production. Methods for assessing the flexibility and readjustability of equipment are considered, which can be used as the basis for the developed methodology for the synthesis of technological elements of modules in conditions of computer-aided design. It is proposed to consider the technological system of the flexible manufacturing module (FMM) of threading, as a system in which transitions from state to state occur under the action of the simplest flows with the parameters of the transition probabilities of a continuous Markov chain. The developed mathematical model, which describes the states of a FMM, taking into account the readjustment of its technological elements, makes it possible to reflect the influence on the operation of the module of the parameters of applications for the changeover of processing modes, a tool, a power threading head, basic elements of a machine tool, a device, a loading device. The structure of the model and the labeled graph of the states of the system can be improved as the number of parameters and characteristics is refined. The solution of the resulting system of equations of final probabilities using the normalization condition allows for given (or experimentally obtained) intensities of arrival and service of changeover requests for FMM of threading, to obtain the values of the probability of non-changeover operation, as well as the probabilities of finding the system in an inoperative state due to the corresponding changeovers. For complete information and an objective assessment of the preferred option for use in FMS conditions, it is also necessary to take into account the stochastic processes occurring in the system under real operating conditions.

АНАЛИЗ И ИССЛЕДОВАНИЕ ЭКСПЛУАТАЦИОННОЙ И ТЕХНОЛОГИЧЕСКОЙ НАДЕЖНОСТИ ОБОРУДОВАНИЯ ДЛЯ ВНУТРЕННЕЙ МЕЛКОРАЗМЕРНОЙ РЕЗЬБООБРАБОТКИ

The article presents the results of analysis and research in the direction of improving technological equipment for internal threading of parts in a flexible automated production. Methods for assessing the operational and technological reliability of a flexible manufacturing module (FMM) of thread processing are considered, which can be used as the basis for the developed method of synthesis of its elements. It is proposed to consider the technological system of the flexible manufacturing module (FMM) of thread processing as a system in which transitions from state to state occur under the action of the simplest flows with parameters of the transition probabilities of a continuous Markov chain. The developed mathematical model describing the state of the FMM taking into account the failures of the functioning of its elements, parametric failures, as well as taking into account the recovery after these types of failures, makes it possible to reflect the influence on the operation of the module of the parameters of the flows of failures and restorations of the tool, machine tool, fixture, loading device. The solution of the obtained systems of equations of final probabilities allows for given (or experimentally obtained) intensities of failure streams (functioning and parametric) for FMM threading to obtain the values of the probability of failure–free operation, as well as the probabilities of finding the system in an inoperative state due to corresponding failures. The measures taken make it possible to solve the synthesis problem at the level of structural and layout optimization, so that at the stage of parametric synthesis to determine the elements that are vulnerable according to the developed criterion, the improvement of which will lead to the creation of the most efficient system.

Convergence of Markov chain transition probabilities

Consider a discrete time Markov chain with rather general state space which has an invariant probability measure $\mu$. There are several sufficient conditions in the literature which guarantee convergence of all or $\mu$-almost all transition probabilities to $\mu$ in the total variation (TV) metric: irreducibility plus aperiodicity, equivalence properties of transition probabilities, or coupling properties. In this work, we review and improve some of these criteria in such a way that they become necessary and sufficient for TV convergence of all respectively $\mu$-almost all transition probabilities. In addition, we discuss so-called generalized couplings.

A Combined Markov Chain and Reinforcement Learning Approach for Powertrain-Specific Driving Cycle Generation

<div class="section abstract"><div class="htmlview paragraph">Driving cycles are valuable tools for emissions calibration at engine and powertrain test beds. While generic velocity profiles were sufficient in the past, legislative changes and increasing complexity of powertrain and exhaust aftertreatment systems require a new approach: Realistically transient cycles - which include critical driving maneuvers and can be tailored to a specific powertrain configuration - are needed to optimize the emission behavior of the said powertrain.</div><div class="htmlview paragraph">For the generation of realistic velocity profiles, the Markov chain approach has been widely used and described in literature. However, this approach, so far, has only been used to generate cycles that are statistically representative of a large database of real driving trips, which is typically not available during the early stages of development of a new powertrain.</div><div class="htmlview paragraph">The work at hand combines a Markov chain approach for driving cycle generation with Q-learning - a reinforcement learning algorithm - to generate driving cycles that meet user-defined criteria and include critical driving maneuvers reflecting individual powertrain development targets.</div><div class="htmlview paragraph">Using simplified models of the powertrain, vehicle and driver and an iterative process, the generated driving cycles are evaluated regarding the fulfillment of the set criteria. The learning algorithm then adapts the Markov chain transition probability matrix to increase the occurrence of the desired critical driving maneuvers and reduce the share of undesired stochastic elements.</div><div class="htmlview paragraph">Exemplary results show that the proposed method is indeed able to generate driving cycles with certain realistic properties that include the desired maneuvers with the expected statistical variance.</div></div>

Situation forecasting and decision-making optimization based on using markov finite chains for areas with industrial pollutions

The paper considers the issues of predicting the situations and optimizing decision-making to improve the environmental situations in the areas with industrial pollution based on the finite Markov’s chains. The article systematizes the existing approaches to forecasting technological risks. The problems associated with the search for optimal forms of environmental safety management and approaches for predicting anthropogenic impact on the environment using mathematical models are considered. To predict the state of the environment, stochastic modeling is proposed, the basis of which is the theory of finite Markov chains. A technique for predicting and optimizing the economic effect on a discrete set of strategies has been developed. The figures show: building system states graph, determining the basic characteristics of system states, finding transition probabilities of Markov chains for non-critical states, a typical cycle of checking the model’s adequacy and system quality. Based on the analysis of existing approaches to forecasting technological risks, a methodology has been developed for forecasting and optimizing the economic effect on a discrete set of strategies. The proposed methodology allows combining economic estimates with the ability to predict the situations and optimize decision-making to improve the environmental situation in the areas of possible chemical pollution. Using the developed methodology will increase the efficiency of the industrial enterprises, facilitate generating informed management decisions, create software and hardware ways to respond the emergencies. The methodology for modeling engineering within nature complex systems and the optimization of decision-making based on finite Markov chains in the areas with industrial pollution will be helpful to researchers and operators of complex technical systems in predicting emergencies using environmental monitoring systems.

Confidence intervals for Markov chain transition probabilities based on next generation sequencing reads data.

Markov chains (MC) have been widely used to model molecular sequences. The estimations of MC transition matrix and confidence intervals of the transition probabilities from long sequence data have been intensively studied in the past decades. In next generation sequencing (NGS), a large amount of short reads are generated. These short reads can overlap and some regions of the genome may not be sequenced resulting in a new type of data. Based on NGS data, the transition probabilities of MC can be estimated by moment estimators. However, the classical asymptotic distribution theory for MC transition probability estimators based on long sequences is no longer valid. In this study, we present the asymptotic distributions of several statistics related to MC based on NGS data. We show that, after scaling by the effective coverage d defined in a previous study by the authors, these statistics based on NGS data approximate to the same distributions as the corresponding statistics for long sequences. We apply the asymptotic properties of these statistics for finding the theoretical confidence regions for MC transition probabilities based on NGS short reads data. We validate our theoretical confidence intervals using both simulated data and real data sets, and compare the results with those by the parametric bootstrap method. We find that the asymptotic distributions of these statistics and the theoretical confidence intervals of transition probabilities based on NGS data given in this study are highly accurate, providing a powerful tool for NGS data analysis.

Robust finite‐time H∞ synchronization for uncertain discrete‐time systems with nonhomogeneous Markovian jump: Observer‐based case

SummaryIn this article, the problem of robust finite‐time H∞ synchronization control is investigated for a class of uncertain discrete‐time master‐slave systems with Markovian switching parameters in the observer‐based case. Parameter uncertainties are assumed to be norm‐bounded, and the polyhedral character is utilized to describe the transition probabilities of nonhomogeneous Markov chain. By using stochastic Lyapunov function method and finite‐time analysis techniques, novel sufficient conditions that include the master‐slave parameters are obtained for designing an observer‐based finite‐time H∞ synchronization control law in terms of linear matrix inequalities. The effectiveness of the proposed theoretical scheme is finally demonstrated by some simulations.

Robust finite-time fault-tolerant control for networked control systems with random delays: A Markovian jump system approach

In this paper, the problems of robust finite-time fault-tolerant stochastic stability and stabilization of networked control systems (NCSs) in the presence of random delays and actuator faults are investigated. First, sensor-to-controller and controller-to-actuator random delays are modeled as a Markov chain. Since in the network environment, the accurate access to the transition probabilities (TPs) of Markov chain is hard or even impossible, TPs information is limited. Then, by considering actuator fault indicator matrix and employing the state augmentation technique, the resulting closed-loop system is transformed into a Markovian jump system (MJS). Based on this model, the sufficient conditions are developed to ensure the finite-time fault-tolerant stochastic stability of the faulty NCS, and the fault-tolerant controllers are designed by solving a feasibility problem in terms of linear matrix inequalities (LMIs). The effectiveness of the proposed approach is illustrated during the simulation of a numerical example. Moreover, the faulty behavior of a continuous-stirred tank reactor (CSTR) is investigated and the applicability of the proposed method is demonstrated.

A weighted $$\chi ^2$$ test to detect the presence of a major change point in non-stationary Markov chains

The problem of detecting a major change point in a stochastic process is often of interest in applications, in particular when the effects of modifications of some external variables, on the process itself, must be identified. We here propose a modification of the classical Pearson $$\chi ^2$$ test to detect the presence of such major change point in the transition probabilities of an inhomogeneous discrete time Markov Chain, taking values in a finite space. The test can be applied also in presence of big identically distributed samples of the Markov Chain under study, which might not be necessarily independent. The test is based on the maximum likelihood estimate of the size of the ’right’ experimental unit, i.e. the units that must be aggregated to filter out the small scale variability of the transition probabilities. We here apply our test both to simulated data and to a real dataset, to study the impact, on farmland uses, of the new Common Agricultural Policy, which entered into force in EU in 2015.