Transition Matrix Of Markov Chain Research Articles

Urban Growth Monitoring and Prediction Using Remote Sensing Urban Monitoring Indices Approach and Integrating CA-Markov Model: A Case Study of Lagos City, Nigeria

As one of the swiftly advancing megacities globally, Lagos faces significant challenges in managing its urban expansion. Mainly, this study focuses on monitoring and predicting urban growth using a comprehensive approach incorporating Global Land 30 (GL30), satellite-based nighttime light observations, and built-up and population density data. The application of remote sensing techniques, combined with utilizing the GL30 dataset, provides an effective means to monitor and predict urban growth trends and patterns. The major patterns occurred from 2000 to 2020, including increased cultivated land; reductions in grasslands, shrublands, and wetlands; and major urbanization. Predictive models indicate that urbanization will continue. Furthermore, employing the Cellular Automata (CA) Markov model in land-use and land-cover (LULC) change prediction. The findings revealed significant changes in LULC over the two decades. Particularly, the percentage of artificial terrain increased from 17.016% to 25.208%, and the area under cultivation increased significantly, rising from 46,771 km2 (1.238%) in 2000 to 75,283 km2 (1.993%) in 2020. Grasslands fell from 7.839% to 1.875%, while forest cover somewhat increased, climbing from 39.319% to 43.081%. Additionally, marshes fell from 9.788% to 5.646%, while shrublands decreased from 4.421% to 2.640%. Surprisingly, bare ground decreased sharply from 0.677% to 0.003%. To forecast future LULC changes, the study also used a Markov Chain Transition Matrix. According to the data, there is a 3.54% chance that agricultural land will become urban, converting it from being used for agriculture to urban development. On the other hand, just 1.05% of forested regions were likely to become municipal areas. This study offers foundations for the upcoming research to enhance urban growth models and sustainability strategies in the face of rising urbanization and environmental concerns in the region, as well as laying the groundwork for informed decision-making in the region.

A multi-layer perceptron–Markov chain based LULC change analysis and prediction using remote sensing data in Prayagraj district, India

Land use land cover (LULC) mapping and temporal observations are indispensable drivers for sustainable development. This research showed the growth trends and land use transition for the Prayagraj district in the last three decades. Supervised classification of Landsat images was performed on 5-year temporal intervals using a maximum likelihood classifier. All satellite images were organized into six major LULC feature classes viz agriculture/open land, barren land, built-up, forest, sand, and water. The overall accuracy of LULC classification was achieved by more than 89% in all seven temporal points. Furthermore, the accuracy of the classified maps was estimated through area-based error matrix. The Land Change Modeler tool of TerrSet 2020 software was used to analyze the transition of classes and to incorporate the multi-layer perceptron-Markov chain (MLP-MC) technique. The transition potentials were included in MLP-MC with the help of sensitive explanatory variables and significant transitions of classes. Furthermore, these transition potentials and the Markov chain transition matrix were used to predict the future LULC dynamics and vulnerability. The change analysis revealed that a significant portion of the agriculture/open land gradually decreased and got converted to built-up land. The results depicted that agriculture/open land was reduced by 8.03% in the last three decades while the built-up region was grown by 199.61%. Forest area was continuously decreasing while the sand area increased due to river meandering. Overall, more than 75% of accuracy was achieved in MLP. The prediction model was first validated with observed data, and then the LULC scenario of 2035 and 2050 was simulated. LULC of 2050 showed that the built-up area would likely reach 13.90% of district area whereas the forest area would remain only 0.79%. The prediction model has given the output in the form of future LULC map along with projected potential transition maps. This would be useful for sustainable urban planning to deal with the alarming rate of built-up growthand agriculture/open landshrinkage.

Python script fuzzy time series Markov chain model for forecasting the number of diseases cocoa plant in Bendungan district

Cocoa is a plantation commodity whose role is essential for the economy, so it is necessary to be aware of its health to maximize production. Cocoa plant disease data is a time series data because it is observed continuously. One of the time series forecasting models is the Fuzzy Time Series Markov Chain (FTS-MC), a combination of the Fuzzy Time Series (FTS) and Markov Chain models. The model uses the principle of fuzzy logic by transferring FTS data to fuzzy logic and using the obtained fuzzy logic groups to derive the Markov chain transition matrix. In this research, a Python script of the FTS-MC model was built in the Google Colaboratory to forecast the number of cocoa plant diseases in Bendungan District to simplify and speed up the data processing. Python was used in this research because of its easy-to-use, flexible, and open-source syntax. In building Python scripts, libraries and functions are needed by utilizing loop processes and if-else statements. Based on the processing results, forecasting with the FTS-MC model using Python only takes less than 1 minute.

Open-flow mixing and transfer operators.

We study finite-time mixing in time-periodic open flow systems. We describe the transport of densities in terms of a transfer operator, which is represented by the transition matrix of a finite-state Markov chain. The transport processes in the open system are organized by the chaotic saddle and its stable and unstable manifolds. We extract these structures directly from leading eigenvectors of the transition matrix. We use different measures to quantify the degree of mixing and show that they give consistent results in parameter studies of two model systems. This article is part of the theme issue 'Mathematical problems in physical fluid dynamics (part 1)'.

Development of an assessment and prediction framework for bridge expansion joints

Expansion joints (EJ)s are considered one of the weakest elements in bridges as their lifetime is considerably shorter than that of other elements. The objective of this research was to select an appropriate system for assessment and maintenance of bridges’ EJs. The system seeks to predict the conditions of EJs, with the aim of offering appropriate maintenance and repair strategies that extend their life spans. An EJ management system (EJMS) was developed that optimizes cost over time, targeting the optimum maintenance strategy for EJs, through visual inspections and a genetic algorithm to construct a unique Markov chain transition matrix capable of predicting EJ deterioration. The model was tested using case studies of existing bridges in Egypt and the results were compared against visual inspection, with the comparison showing deviations between 3.2% and 8.1%. The EJMS can help decision makers in selecting the optimum maintenance strategy that maximizes the overall conditions of EJs while meeting budgetary constraints.

Keeping random walks safe from extinction and overpopulation in the presence of life-taking disasters

ABSTRACT Recurrence and transience conditions are made explicit in discrete-time Markov chain population models for which random stationary growth alternates with disastrous random life-taking events. These events either have moderate stationary magnitudes or lead to an abrupt population decline. The probability of their occurrence may or may not depend on the population size. These conditions are based on the existence or not of a “weak” carrying capacity, where “weak” means that the carrying capacity can be exceeded, temporarily. In this framework, the population is threatened with extinction, an event whose probability is expressed, as well as the law of the time remaining until this deadline. On the other hand, the population is also threatened by overpopulation, an event whose time to reach a given threshold is expressed, as well as the difference between the population size and the carrying capacity. The theory is that of extreme values for Markov chains and is based on the control of the spectral properties of the northwest truncation of the transition matrix of the original Markov chain with life-taking disasters. The article presents an extension to the case where the process of life-taking disasters is no longer geometric and to the case where the probability of occurrence of a disaster depends on the population size. Both the time to extinction and the time to a given threshold have geometrically decaying distribution tails. The use of the extremal Markov chain in the calculation of the time to overpopulation is innovative.

Markov chains to determine the probability of climate change for planting selection in the city of Caxias do Sul

ABSTRACT: The Markov stochastic chain model and the analytical hierarchy process (AHP) were used as tools to support decision-making for the best crop-planting choice in the city of Caxias do Sul, Brazil. Temperature and precipitation information were collected from the Meteorological Database for Teaching and Research of the National Institute of Meteorology of Brazil for the period 1997-2017. The stochastic model was applied to obtain the probability of transition between a range of variations for temperature and precipitation. In the second phase of the study, an algebraic model was developed, making it possible to link the probability of the Markov chain transition matrix to the AHP judgment matrix. In the third phase, the AHP was applied as a tool to determine the most beneficial crop that could be planted for the studied city, considering the evaluated criteria: temperature, precipitation, and soil pH. The alternatives for crop planting were carrots, tomatoes, apples, and grapes. These were chosen because they are the most-planted crops in the city of Caxias do Sul. The ranking of the benefit-force results of applying the model for spring was carrots (0.297), apples (0.259), grapes (0.228), and tomatoes (0.215); for summer: grapes (0.261), tomatoes (0.261), apples (0.238), and carrots (0.230); for autumn: carrots (0.316), grapes (0.243), tomatoes (0.228), and apples (0.213); and for winter: carrots (0.327), tomatoes (0.235), apples (0.222), and grapes (0.216). Thus, it was concluded that farmers would have a better chance of success if they planted carrots during the spring, autumn, and winter, and grapes during the summer.

Open Markov Type Population Models: From Discrete to Continuous Time

We address the problem of finding a natural continuous time Markov type process—in open populations—that best captures the information provided by an open Markov chain in discrete time which is usually the sole possible observation from data. Given the open discrete time Markov chain, we single out two main approaches: In the first one, we consider a calibration procedure of a continuous time Markov process using a transition matrix of a discrete time Markov chain and we show that, when the discrete time transition matrix is embeddable in a continuous time one, the calibration problem has optimal solutions. In the second approach, we consider semi-Markov processes—and open Markov schemes—and we propose a direct extension from the discrete time theory to the continuous time one by using a known structure representation result for semi-Markov processes that decomposes the process as a sum of terms given by the products of the random variables of a discrete time Markov chain by time functions built from an adequate increasing sequence of stopping times.

Variable structure interacting multiple model for highly aperiodic sensors with poor measurement precision

The issue of aircraft tracking using passive sensors with poor measurement precision and a variable update rate is investigated. First, the need for a time-dependent Markov chain transition matrix for the Interacting Multiple Model (IMM) is thoroughly evaluated using simulation and real data from an experimental multilateration system. Three different methods for computing the time-dependent matrix are compared, showing their superiority for highly aperiodic sensors. Second, the Variable Structure Interacting Multiple Model (VS-IMM) for passive sensors with poor geometry is proposed, in which the mode set update is based on local dilution of precision or distance root mean square error. The performance of the estimator is compared with the IMM using real data. The VS-IMM results in the best performance for areas of both high and low measurement precision.

Using the Markov chain to predict the outcome of an election in a given situation

This paper used Markov Chain, a mathematical model that is often utilized in predicting, to forecast the result of the election process of a person in a specific circumstance. In the way of modeling, starting from the tree diagram to the model of Markov Chain and related transition matrix, this paper aimed at the election result, which relates the object of study, a specific candidate. The research data are collected from the prompt of the research question and the calculation of coding based on the transition matrix. The result is, when it is the 64th year of that specific candidate running for the reelection campaign, he will be likely to retire from politics.

Quasi-metrics on graphs

In this paper we consider some questions of the theory and practice of mean first passage time quasi-metric, a generalized metric structure closely related to ergodic homogeneous Markov chains. In particular, we consider the structure and properties of mean first passage time quasimetrics based on several graph models, including simple cycles, simple paths and their oriented analogues. The introduction contains the history of the problem and provides an overview of the main ideas and results presented in the article. The second section gives basic concepts of the theory of Markov chains. In fact, a Markov chain is a mathematical model of some random process describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. This section collects basic definitions needed to consider the role of graph models in the presentation and study of ergodic homogeneous Markov chains. The Markov chain can be depicted as an oriented weighted graph of transitions whose vertices correspond to the states of the chain and the arcs correspond to the transitions between them. The process will be ergodic if this weighted oriented graph is weakly connected, and the largest common divisor of the lengths of all its cycles is equal to 1. On the other hand, any connected graph can be used as a basis for building a model of the simplest Markov chain: if a vertex 𝑖 has degree 𝑘, all incident edges are converted into arcs with the weights 1/𝑘 . Moreover, in the second section the definition of the mean first passage time for an ergodic homogeneous Markov chain is given. The algorithm of finding the mean first passage time is analyzed in detail by using converging trees of the oriented graph, related to the transition matrix of the ergodic homogeneous Markov chain. At last, a mean first passage time is analyzed as the quasi-metric 𝑚 of mean first passage time on the vertices 𝑉 = {1, 2, ..., 𝑛} of the oriented graph corresponding to the transition matrix of a given ergodic homogeneous Markov chain: 𝑚(𝑖, 𝑗) is the expected number of steps (arcs) for random wandering on the oriented graph Γ, starting at 𝑖, to reach 𝑗 for the first time. This quasi-metric has a number of important theoretical and applied properties. The third section deals with the construction and research of mean first passage time quasimetrics for the undirected cycles 𝐶𝑛, 𝑛 ≥ 3. Examples of constructions of mean first passage time quasi-metrics of undirected cycles for small values of 𝑛 are considered. Illustrations of the ”graphical“ procedure of building the matrix 𝑀 are given. Properties of the resulting generalized metric structures are analyzed. In the fourth section, similar considerations for mean first passage time quasi-metrics of the undirected paths 𝑃𝑛, 𝑛 ≥ 2, are represented. The fifth section deals with the construction and research of mean first passage time quasimetrics for the directed (oriented) cycles 𝐶𝑛, 𝑛 ≥ 3. Examples of constructions of mean first passage time quasi-metrics of undirected (oriented) cycles for small values of 𝑛 are considered. Illustrations of the ”graphical“ procedure of building the matrix 𝑀 are given. Properties of the resulting generalized metric structures are analyzed. In the sixth section, similar considerations for mean first passage time quasi-metrics of the directed (oriented) paths 𝑃𝑛, 𝑛 ≥ 2, are represented. In the conclusion, the results of the work are summed up and possible directions of further research are outlined.

A robust optimization approach for protecting power systems against cascading blackouts

Due to uncertainties and the complicated intrinsic dynamics of power systems, it is difficult to predict the cascading failure paths once the cascades occur. This makes it challenging to achieve the effective power system protection against cascading blackouts. By incorporating uncertainties and stochastic factors of the cascades, a Markov chain model is developed in this paper to predict the cascading failure paths of power systems. The transition matrix of Markov chain is dependent on the probability of branch outage caused by overloads or stochastic factors. Moreover, a robust optimization formulation is proposed to prevent the cascading blackouts by optimal load shedding and generation control for multiple cascading failure paths with relatively high probabilities. Since each state on the cascading failure paths can be described by one convex set, the proposed robust optimization problem is equivalent to the best approximation problem in Euclidean space. Thus, an efficient numerical solver based on Dykstra’s algorithm is employed to deal with the robust optimization problem. In theory, we provide a lower bound for the probability of preventing the cascading blackouts of power systems. Finally, the proposed approach for power system protection is verified by a case study of IEEE 118 bus system.

О специальных свойствах некоторых квазиметрик

В статье рассмотрены свойства квазиметрики среднего времени первого прохода - обобщенной метрической структуры, тесно связанной с эргодическими однородными цепями Маркова.Во введении представлена история вопроса, дан обзор основных идей и результатов работы.В первом разделе собраны основные понятия теориицепей Маркова - последовательностей случайных событий с конечным или счетным числом исходов, характеризующихся тем, что, говоря нестрого, при фиксированномнастоящем будущее независимо от прошлого. Точнее, математическая модель некоторого случайного процесса представляет собой марковскую цепь, если распределение вероятностей параметров процесса в следующий момент времени зависиттолько от параметров процесса в предыдущий момент.Во втором разделе собраны базовые определения, необходимые для рассмотрения роли графовых моделейв представлении и исследовании эргодических однородных цепей Маркова.Марковская цепь может быть изображена в виде ориентированного взвешенного графа переходов, вершины которого соответствуют состояниям цепи, а дуги - переходам между ними.Процесс будет эргодическим, если построенный взвешенный орграф является слабо связным, и наибольший общий делитель длин всех его циклов равен единице.С другой стороны, любой связный граф может служить базой для построения модели простейшей цепи Маркова: если вершина $i$ имеетстепень $k$, то все выходящие из нее ребра превращаются в дуги с весами $\frac{1}{k}$.%Основное место занимают понятия, связанные с теорией остовного леса сходящихся деревьев ориентированного %графа, соответствующего матрице перехода однородной эргодической цепи Маркова.В третьем разделе дано определение среднего времени первого прохода для однородной эргодической цепи Маркова. Представлены несколько способов построения соответствующей матрицы $M$. Подробно проанализирован алгоритм нахождения среднего времени первого прохода с помощью использования сходящихся деревьев ориентированного графа, связанного с матрицей перехода эргодической однородной цепи Маркова. Описана родственная рекуррентная процедура.В четвертом разделе матрица среднего времени первого прохода рассмотрена как {\it квазиметрика } $m$среднего времени первого прохода на множестве вершин $V=\{1, 2, ..., n\}$ ориентированного графа, соответствующего матрице перехода эргодической однородной цепи Маркова: $m(i,j)$ - ожидаемое количество шагов (дуг) для случайного блуждания на орграфе $\Gamma$, начинающегося с $i$, для достижения $j$ в первый раз. Эта квазиметрика обладает рядом важных теоретических и прикладных свойств.В частности, квазиметрика среднего времени первого прохода для простого случайного блуждания по связному невзвешенному графу $G$, в котором из любой вершины графа существует равная вероятность перемещения в любую соседнюю вершину,является взвешиваемой квазиметрикой, т.е. существует весовая функция $w: V\rightarrow\mathbb{R}_{\ge 0}$,%$w=(w_1, w_2,\ldots, w_n)$,такая, что для всех $i,j\in V$ имеет место сотношение %& Было бы здорово найти для $m_i$ наглядную интерпретацию.$m(i,j)+w_i=m(j,i)+w_j.$Менее изучены, но не менее интересны связи квазиметрики среднего времени первого прохода с другими метрическими структурами на графах, в частности, с $\alpha$-метрикой леса и ее вариациями.Наконец, в пятом разделе рассмотрены примеры построения и исследования квазиметрики среднего времени первого прохода. Помимо иллюстрации ''графовой`` процедуры построения матрицы $M$, представлены рекуррентные алгоритмы исследования и проанализированы получающиеся при этом обобщенные метрические структуры.

A novel fuzzy-Markov forecasting model for stock fluctuation time series

In order to reveal intrinsic fluctuation rules hidden in a stock market time series dataset with noise, a novel forecasting model combining Markov chain theory with fuzzy set theory is proposed in this study. A fuzzified one-step transition matrix of Markov Chain in the paper represents inherent rules of historical fluctuation. Comparing with existing models, the advantage of the proposed model is that transition matrix can express the relationship between history and current flexibly while the introduction of fuzzy theory can help to alleviate noises. Therefore, the proposed model could handle complex patterns during state transitions and the relatively simple forecasting algorithm could reduce the calculation cost. We apply the proposed method to forecast well-known stock indexes such as (Taiwan Stock Exchange Capitalization Weighted Stock Index) TAIEX, (Shanghai Stock Exchange Composite Index) SHSECI and so on. Experimental results demonstrate that our proposed method outperforms other traditional models.

Quantum fast-forwarding: Markov chains and graph property testing

We introduce a new tool for quantum algorithms called quantum fast-forwarding (QFF). The tool uses quantum walks as a means to quadratically fast-forward a reversible Markov chain. More specifically, with P the Markov chain transition matrix and D = \sqrt{P\circ P^T} its discriminant matrix (D=P if P is symmetric), we construct a quantum walk algorithm that for any quantum state |v> and integer t returns a quantum state \epsilon-close to the state D^t|v>/\|D^t|v>. The algorithm uses O(|D^t|v>|^{-1}\sqrt{t\log(\epsilon\|D^t|v>})^{-1}}) expected quantum walk steps and O(\|D^t|v>|^{-1}) expected reflections around |v>. This shows that quantum walks can accelerate the transient dynamics of Markov chains, complementing the line of results that proves the acceleration of their limit behavior. We show that this tool leads to speedups on random walk algorithms in a very natural way. Specifically we consider random walk algorithms for testing the graph expansion and clusterability, and show that we can quadratically improve the dependency of the classical property testers on the random walk runtime. Moreover, our quantum algorithm exponentially improves the space complexity of the classical tester to logarithmic. As a subroutine of independent interest, we use QFF for determining whether a given pair of nodes lies in the same cluster or in separate clusters. This solves a robust version of s-t connectivity, relevant in a learning context for classifying objects among a set of examples. The different algorithms crucially rely on the quantum speedup of the transient behavior of random walks.

Optimal Maneuver Strategy of Observer for Bearing-Only Tracking in Threat Environment

The optimal maneuver of observer for bearing-only tracking (BOT) in a threat environment is a complex problem which involves nonlinear filtering, threat avoidance, and optimal maneuver strategy. Under comprehensive consideration, the reward function comprised of the lower bound on detFIM and threat cost was established; the finite-horizon MDP principle was applied to obtain the optimal strategy. The quantization method was used to discretize the BOT process and calculate the transition matrix of Markov chain; to achieve quantization in the beginning of each period, CKF was applied to provide the initial state estimate and the corresponding error covariance. The numerical simulations illustrated the applicability and superior performance for static and dynamic target tracking in several scenarios in the threat environment.

Error bounds for augmented truncation approximations of Markov chains via the perturbation method

AbstractLetPbe the transition matrix of a positive recurrent Markov chain on the integers with invariant probability vectorπT, and let(n)P̃ be a stochastic matrix, formed by augmenting the entries of the (n+ 1) x (n+ 1) northwest corner truncation ofParbitrarily, with invariant probability vector(n)πT. We derive computableV-norm bounds on the error betweenπTand(n)πTin terms of the perturbation method from three different aspects: the Poisson equation, the residual matrix, and the norm ergodicity coefficient, which we prove to be effective by showing that they converge to 0 asntends to ∞ under suitable conditions. We illustrate our results through several examples. Comparing our error bounds with the ones of Tweedie (1998), we see that our bounds are more applicable and accurate. Moreover, we also consider possible extensions of our results to continuous-time Markov chains.

Estudio estocástico con el uso de cadenas de Markov para la transmisión del dengue

El dengue es una enfermedad viral transmitida a los seres humanos por picaduras del mosquito Aedes aegypti, que puede conducir a diferentes estadios de la enfermedad. Esta enfermedad se ha extendido a gran parte del mundo y afecta a millones de personas. En este trabajo se presenta un estudio basado en la matriz de transición de una cadena de Markov homogénea, donde sus estados coinciden con las posibles etapas por las cuales una persona transita en la dinámica de transmisión del dengue y el comportamiento de las entidades que varían en el tiempo. También propone una estrategia novedosa de control basada en la ponderación de las probabilidades trascendentales incorporándolo a la matriz con el objetivo de reducir el impacto de la epidemia en la sociedad. La idea de ponderar determinadas probabilidades con estrategias de control brinda opciones para los sistemas de salud en el control de epidemias y el mejoramiento de las calidad de vida de los seres humanos, con una herramienta como las cadenas de Markov, cuyo éxito radica en que es suficientemente complejo para describir ciertas características no triviales de algunos sistemas, pero al mismo tiempo fácil de analizar matemáticamente

Distributed H∞ State Estimation Over a Filtering Network With Time-Varying and Switching Topology and Partial Information Exchange.

This paper is concerned with the distributed H∞ state estimation for a discrete-time target linear system over a filtering network with time-varying and switching topology and partial information exchange. Both filtering network topology switching and partial information exchange between filters are simultaneously considered in the filter design. The topology under consideration evolves not only over time but also by an event switch which is assumed to be subject to a nonhomogeneous Markov chain. The probability transition matrix of the nonhomogeneous Markov chain is time-varying. In the filter information exchange, partial state estimation information and channel noise are simultaneously considered. In order to design such a switching filtering network with partial information exchange, stochastic Markov stability theory is developed. The switching topology-dependent filters are derived to guarantee an optimal H∞ disturbance rejection attenuation level for the estimation disagreement of the filtering network. It is shown that the addressed H∞ state estimation problem is turned into a switching topology-dependent optimal problem. The distributed filtering problem with complete information exchanges from its neighbors is also investigated. An illustrative example is given to show the applicability of the obtained results.

Uniformization: Basics, extensions and applications

Uniformization, also referred to as randomization, is a well-known performance evaluation technique to model and analyse continuous-time Markov chains via an easier to performance measures via iteration of the one-step transition matrix of the discrete-time Markov chain. The number of iterations has a Poisson distribution with rate dominating the maximum exit rate from the states of the continuous-time Markov chain. This paper contains an expository presentation of uniformization techniques to increase awareness and to provide a formal and intuitive justification of several exact and approximate extensions, including: •exact uniformization for reward models,•exact uniformization for time-inhomogeneous rates,•a numerical comparison with simple time-discretization,•approximate uniformization for unbounded transition rates, and•exact uniformization for continuous state variables for non-exponential networks.Furthermore, several of these results are numerically illustrated for a processor sharing web server tandem model of practical interest.