Concept Of Markov Chains Research Articles

Markov chain analysis to predict natural disasters in the Province of Bangka Belitung Islands as part of preventative measures to prevent environmental damage

Bangka Belitung Islands Province is a province which is famous for its beauty and natural wealth. In addition to its natural beauty and wealth, this province has disaster-prone areas. Based on the results of the 2021 Indonesian Disaster Risk Index (IRBI) measurement, the Bangka Belitung Islands Province has the second highest risk class in Indonesia with a value of 160.98. Therefore, a study is needed that aims to determine the prediction of natural disasters in the Bangka Belitung Islands Province as a preventive measure to prevent environmental damage. The analytical method used in this research is the Markov Chain method. The data used in this study is secondary data obtained from the official website of the National Disaster Management Agency (BNPB) from 2017 to 2022 with several types of disasters, including floods, landslides, tornadoes, forest and land fires, and natural disasters others (missing persons, house fires, drowning persons, etc.). Based on the results of calculations using the Markov Chain concept, it can be concluded that the possibility of flooding in 2023 is 31.46%, landslides are 7.73%, tornadoes are 43.81%, forest and land fires are 14.32%, and other disasters of 2.69% with a Mean Absolute Error (MAE) of 0.356 and a Mean Square Error (MSE) of 0.17.

Research on the application of Markov chain

In modern times, mathematicians are often troubled by new approaches to deducing the outcomes of certain events. The introduction of Markov models eliminated queries across different sectors. In 1907, Russian mathematician Andrey Markov proposed the concept of Markov chains. It has been widely used in many aspects, such as weather prediction, deep learning, biological information, and so on. Therefore, this paper examines how the Markov chain model can be applied in a variety of situations. This study uses Python as a supporting tool to simulate states and possible outcomes. It can be concluded that Python is able to simulate the state transitions of the Markov chain. The paper also identifies the differences between Markov models, their application in common scenarios such as medical, finance, weather forecasting, machine learning and others in our everyday life and why they are so popularly used, including the simplicity of the model and more.

Integrated Color and Intensity MSF Scheme for Image Retrieval Systems

Markov Stationary Features (MSF) based on homogeneous Markov chain model, for content-based image analysis, is getting popularity nowadays. It not only considers the distribution of colors, just as the histogram method does, but also characterizes the spatial co-occurrence of histogram patterns. However, handling a large-scale database of images with a degree of heterogeneity, a simple MSF method is not sufficient to discriminate the images as one requires. In this paper, an Integrated Color and Intensity MSF (ICI-MSF) based on non-homogeneous Markov Chain is proposed to overcome this shortcoming. By incorporating spatial co-occurrence of image intensities with the spatial co-occurrence information of colors and exploiting time inhomogeneous Markov chain concept, it is possible to improve certain aspects of the existing methods. Without compromising effectiveness and robustness, the method proposed in this paper keeps the feature level simplicity. Widely recognized databases namely WANG1000 and Corel10800, are used to evaluate and compare the performance of the proposed algorithm with the existing methods. The experimental results justify the effectiveness of the proposed method.

APPLICATION OF RENEWAL REWARD PROCESSES IN HOMOGENEOUS DISCRETE MARKOV CHAIN

A renewal process which is a special type of a counting process, which counts the number of events that occur up to (and including) time has been investigated, in order to provide some insight into the performance measures in renewal process and sequence such as, the mean time between successive renewals, ; Laplace-Stiltjes transform (LST) of the mean time, ; the Laplace-Stieltjes transform (LST) of the mean time distribution function, ; Laplace-Stiltjes transform (LST) of fold convolution of distribution function, ; the time at which the renewal occurs, the average number of renewals per unit time over the interval (0, t], and expected reward, . Our quest is to analyse the distribution function of the renewal process and sequence using the concept of discrete time Markov chain to obtain the aforementioned performance measures. Some properties of the Erlang , exponential and geometric distributions are used with the help of some existing laws, theorems and formulas of Markov chain. We concluded our study through illustrative examples that, it is not possible for an infinite number of renewals to occur in a finite period of time; Also, the expected number of renewals increases linearly with time; and from the uniqueness property, we affirmed that, the Poisson process is the only renewal process with a linear mean-value function; and lastly, we obtained the optimal replacement policy for a manufacturing machine which showed that, the exponential property of the lifetime

A hybrid spectrum access approach for efficient channel allocation and power control in cognitive radio networks

SummaryRadio spectrum is the backbone for wireless communication. Many recent studies show that most of the available spectrum has already been allocated. Cognitive radio (CR) provides a new dimension in the field of wireless communication and is considered as a life line for the spectrum scarcity problem as seen all around the world nowadays. CR enables the use of dynamic spectrum access, where secondary users (SUs) or unlicensed users are allowed to opportunistically use the spectrum when it is not being used by the primary users (PUs) or licensed users. Medium Access Control (MAC) protocols use the sensing capability and the primary users behavior to generate the transmission opportunities for SUs. This article proposes two Markov based hybrid access methods namely Hybrid‐P1 and Hybrid‐P2, where a secondary user (SU) dynamically decides it's transmission mode between overlay and underlay depending upon the primary users's (PU's) past and present behavior. Analytical results based on the Markov chain concept are very encouraging and verified through OMNeT++ simulator. This work also analyses the performance of overlay only and underlay only methods, which are well known data transmission techniques in the cognitive radio networks (CRNs). Simulation results show the 47% and 59% performance improvement in Hybrid‐P1 access method, when compared to the overlay only and underlay only access methods respectively. Furthermore simulation results also show the 54% and 65% performance improvement in Hybrid‐P2 access method, when compared to the overlay only and underlay only access methods respectively.

Multisensor and Multiresolution Remote Sensing Image Classification through a Causal Hierarchical Markov Framework and Decision Tree Ensembles

In this paper, a hierarchical probabilistic graphical model is proposed to tackle joint classification of multiresolution and multisensor remote sensing images of the same scene. This problem is crucial in the study of satellite imagery and jointly involves multiresolution and multisensor image fusion. The proposed framework consists of a hierarchical Markov model with a quadtree structure to model information contained in different spatial scales, a planar Markov model to account for contextual spatial information at each resolution, and decision tree ensembles for pixelwise modeling. This probabilistic graphical model and its topology are especially fit for application to very high resolution (VHR) image data. The theoretical properties of the proposed model are analyzed: the causality of the whole framework is mathematically proved, granting the use of time-efficient inference algorithms such as the marginal posterior mode criterion, which is non-iterative when applied to quadtree structures. This is mostly advantageous for classification methods linked to multiresolution tasks formulated on hierarchical Markov models. Within the proposed framework, two multimodal classification algorithms are developed, that incorporate Markov mesh and spatial Markov chain concepts. The results obtained in the experimental validation conducted with two datasets containing VHR multispectral, panchromatic, and radar satellite images, verify the effectiveness of the proposed framework. The proposed approach is also compared to previous methods that are based on alternate strategies for multimodal fusion.

Dynamic Relocations in Car-Sharing Networks

We propose a novel dynamic car relocation policy for a car-sharing network with centralized control and uncertain, unbalanced demand. The policy is derived from a reformulation of the linear programming fluid model approximation of the dynamic problem. We project the full-dimensional fluid approximation onto the lower-dimensional space of relocation decisions only. This projection results in a characterization of the problem as n+1 linear programs, where n is the number of nodes in the network. The reformulation uncovers structural properties that are interpretable using absorbing Markov chain concepts and allows us to write the gradient with respect to the relocation decisions in closed form. Our policy exploits these gradients to make dynamic car relocation decisions. We provide extensive numerical results where our dynamic car relocation policy consistently outperforms the standard static policy in realistic instances from the literature. In fact, it reduces the static policy's optimality gap by more than 13% in all of these instances and up to 35% in some.

Estimating lifetime malnourished period and its statistics based on the concept of Markov chain with reward

Malnutrition among women, accessed through body mass index, has great consequences for achieving key national targets. This study introduces the concept of lifetime malnourished period (LMP): the number of years a woman would remain malnourished, either as underweight or overweight given that she is currently malnourished, and its measures of variation. Markov chain with rewards was used to compute the moments of LMP based on age-specific mortality rates and proportion of women of reproductive age that were either underweight or overweight using data from the 2013 Nigeria Demographic and Health Survey. Each of the two malnutrition status was treated as a Bernoulli-distributed reward with probability taken as the proportion of overweight or underweight women at specific age. Findings indicate that the average LMP for an underweight woman in Nigeria at age 15 years is 2.3 years but 5.8 for overweight. The remaining LMP for underweight is lower among women who attain higher level of education than for those with no or primary level of education with standard deviation reducing with age. Further, we found overweight women from the richest households and those from urban areas to have longer years of remaining in that state of health than their other counterparts, and that longevity contributes more to the variance in LMP for overweight than for underweight women.

An integrated Bayesian–Markovian framework for ascertaining cost of executing quality improvement programs in manufacturing industry

PurposeTypically, the budgetary requirements for executing a supplier’s process quality improvement program are often done in unstructured ways in that quality improvement managers purely use their previous experiences and pertinent historical information. In this backdrop, the purpose of this paper is to ascertain the expected cost of carrying out suppliers’ process quality improvement programs that are driven by original equipment manufacturers (OEMs).Design/methodology/approachUsing inputs from experts who had prior experience executing suppliers’ quality improvement programs and employing the Bayesian theory, transition probabilities to various quality levels from an initial quality level are ascertained. Thereafter, the Markov chain concept enables the authors to determine steady-state probabilities. These steady-state probabilities in conjunction with quality level cost coefficients yield the expected cost of quality improvement programs.FindingsThe novel method devised in this research is a key contribution of the work. Furthermore, various implications related to experts’ inputs, dynamics related to Markov chain, etc., are discussed. The method is illustrated using a real life of automotive industry in India.Originality/valueThe research contributes to the extant literature in that a new method of determining the expected cost of quality improvement is proposed. Furthermore, the method would be of value to OEMs and suppliers wherein the quality levels at a given time are the function of quality levels in preceding period(s).

Framework for Onboard Bus Comfort Level Predictions Using the Markov Chain Concept

Efficiently functioning public transport has a significant positive impact on the entire transportation system performance through numerous aspects, such as the reduction of congestion, energy consumption, and emissions. In most cases, the basic elements of public transport are the bus transport subsystem. Currently, in addition to criteria such as punctuality, the frequency of departures, and the number of transfers, a travelling comfort level is an important element for passengers. An overcrowded bus may discourage travelers from choosing this mode of transport and induce them to use a private car despite the existence of many other facilities offered by a given public transport system. Therefore, the forecasting of bus passenger demand, as well as bus occupancy at individual bus stops, is currently an important research direction. The main goal of the article is to present the conceptual framework for the Advanced Travel Information System with the prediction module. The proposed approach assumes that the prediction module is based on the use of the Markov Chain concept. The efficiency and accuracy of the obtained prediction were presented based on a real-life example, where the measurements of passengers boarding and alighting at bus stops were made in a selected Cracow bus line. The methodology presented in the paper and the obtained results can significantly contribute to the development of solutions and systems for a better management as well as a cost and energy consumption optimisation in the public transport system. Current and forecasted information related to bus occupancy, when properly used in the travel information system, may have a positive impact on the development of urban mobility patterns by encouraging the use of public transport. This article addresses the current and practical research problem using an adequate theoretical mathematical tool to describe it, reflecting the characteristics and nature of the phenomenon being studied. To the best of the authors’ knowledge, the article deals for the first time with the problem of prediction of onboard bus comfort levels based on in-vehicle occupancy.

Bathtub curve as a Markovian process to describe the reliability of repairable components

This study presents a novel mathematical formulation to describe repairable components reliability model based on their bathtub curve and repair rate behaviour. The model is derived from the concept of Markov chain, which allows defining component's lifetime process. In addition, the formulation brings components' degradation quantification. The proposed approach presents a pathway to develop an accurate reliability model for reliability assessments as shown in the presented case study.

Markov chains and rough sets

In this paper, we present a link between markov chains and rough sets. A rough approximation framework (RAF) gives a set of approximations for a subset of universe. Rough approximations using a collection of reference points gives rise to a RAF. We use the concept of markov chains and introduce the notion of a Markov rough approximation framework (MRAF), wherein a probability distribution function is obtained corresponding to a set of rough approximations. MRAF supplements well-known multi-attribute decision-making methods like TOPSIS and VIKOR in choosing initial weights for the decision criteria. Further, MRAF creates a natural route for deeper analysis of data which is very useful when the values of the ranked alternatives are close to each other. We give an extension to Pawlak's decision algorithm and illustrate the idea of MRAF with explicit example from telecommunication networks.

Modeling long correlation times using additive binary Markov chains: Applications to wind generation time series

Wind power generation exhibits a strong temporal variability, which is crucial for system integration in highly renewable power systems. Different methods exist to simulate wind power generation but they often cannot represent the crucial temporal fluctuations properly. We apply the concept of additive binary Markov chains to model a wind generation time series consisting of two states: periods of high and low wind generation. The only input parameter for this model is the empirical autocorrelation function. The two-state model is readily extended to stochastically reproduce the actual generation per period. To evaluate the additive binary Markov chain method, we introduce a coarse model of the electric power system to derive backup and storage needs. We find that the temporal correlations of wind power generation, the backup need as a function of the storage capacity, and the resting time distribution of high and low wind events for different shares of wind generation can be reconstructed.

Evaluating statistically constrained economic design for monitoring the average number of non-conformities with adaptive sampling schemes

Recent researches have shown that control charts with adaptive sampling schemes are faster than the static control charts in detecting process shifts. In this article, based on the Markov chain concepts, adaptive sampling schemes of U control chart are designed for monitoring the average number of non-conformities per unit in the process. Also, the economic statistical design of U control chart with two cost functions has been developed. These economic models considered features such as the cost of false alarms, the cost of detecting assignable causes and repairing, the cost of producing defective items when the process is in-control and out-of-control states and the cost of sampling and inspection. At the end, numerical examples for comparison and demonstrating the applicability of purposed schemes are solved. The results show the significant improvement of the adaptive sampling schemes as compared to the classic scheme based upon the statistical and cost criteria.

Evaluating statistically constrained economic design for monitoring the average number of non-conformities with adaptive sampling schemes

Recent researches have shown that control charts with adaptive sampling schemes are faster than the static control charts in detecting process shifts. In this article, based on the Markov chain concepts, adaptive sampling schemes of U control chart are designed for monitoring the average number of non-conformities per unit in the process. Also, the economic statistical design of U control chart with two cost functions has been developed. These economic models considered features such as the cost of false alarms, the cost of detecting assignable causes and repairing, the cost of producing defective items when the process is in-control and out-of-control states and the cost of sampling and inspection. At the end, numerical examples for comparison and demonstrating the applicability of purposed schemes are solved. The results show the significant improvement of the adaptive sampling schemes as compared to the classic scheme based upon the statistical and cost criteria.

Communication Networks: A Concise Introduction, Second Edition

Abstract This book results from many years of teaching an upper division course on communication networks in the EECS department at the University of California, Berkeley. It is motivated by the perceived need for an easily accessible textbook that puts emphasis on the core concepts behind current and next generation networks. After an overview of how today's Internet works and a discussion of the main principles behind its architecture, we discuss the key ideas behind Ethernet, WiFi networks, routing, internetworking, and TCP. To make the book as self-contained as possible, brief discussions of probability and Markov chain concepts are included in the appendices. This is followed by a brief discussion of mathematical models that provide insight into the operations of network protocols. Next, the main ideas behind the new generation of wireless networks based on LTE, and the notion of QoS are presented. A concise discussion of the physical layer technologies underlying various networks is also included. F...

Strong Laws of Large Numbers and the Asymptotic Equipartition Property for the Asymptotic N-Branch Markov Chains Indexed by a Cayley Tree

We introduce a concept of asymptotic N-branch Markov chains indexed by a Cayley tree. Then we study the strong law of large numbers and the asymptotic equipartition property for the introduced Markov chains with a finite state space, which generalizes a class of results obtained for inhomogeneous Markov chains indexed by a Cayley tree and nonsymmetric Markov chains indexed by a binary tree.

Comparative analysis of predictive methods for drought durations: a case of monthly and annual streamflow droughts in Atlantic Canada

ABSTRACTMethods based on the recursive probability, the extreme number theorem, and Markov chain (MC) concepts were applied to predict drought lengths (duration) on the standardized (termed as standardized hydrological index, SHI) sequences of monthly and annual river flows from Atlantic Canada. Results of the study indicated that the MC-based method is the most efficient, reliable and versatile method for predicting drought durations followed by the extreme-number-based method. The recursive-probability-based method was found to be computationally intensive and less efficient, although it provided a powerful means for calibrating the empirical plotting position formula needed in the MC-based method. The Weibull plotting position formula turned out to be a suitable measure of the exceedance probability in MC methodology for predicting drought lengths in Atlantic Canada. Based on results, it can be inferred that the MC-based method can be extended to MC2 and higher-order chains for predicting drought lengths on SHI sequences. The predictive capability of the extreme-number-theorem-based method is limited only to independent or weakly first-order persistent SHI sequences.EDITOR D. KoutsoyiannisASSOCIATE EDITOR Q. Zhang

Evaluating the reliability of multi-energy source buildings: A new analytical method for considering the dynamic behavior of thermal loads

Developments in energy conversion technology and multi-carrier energy systems have led to higher flexibility of energy systems, thus improving the reliability of supply and decreasing the energy costs. Usually in previous studies on reliability evaluation of Multi-Energy Source Building (MESB), the required energy for supplying the loads is considered to be fixed during each time interval and the ability of the system to supply the loads is evaluated on the basis of this assumption. In this paper, due to the high amount of the thermal loads in a building's energy portfolio, the flexibility of these loads is addressed and an analytical method is presented to model the dynamic behavior of thermal loads in reliability analyses of MCEBs. The proposed method is based on Markov chain concepts integrating thermodynamic equations. In addition, in order to evaluate the validity of the proposed method, a Monte-Carlo simulation method is employed. The methods are simulated on test systems and the results are presented and discussed.

An Adaptive WLAN Interference Mitigation Scheme for ZigBee Sensor Networks

We propose an adaptive interference avoidance scheme that enhances the performance of ZigBee networks by adapting ZigBees' transmissions to measured wireless local area network (WLAN) interference. Our proposed algorithm is based on a stochastic analysis of ZigBee operation that is interfered with by WLAN transmission, given ZigBee and WLAN channels are overlaid in the industrial, scientific, and medical (ISM) band. We assume that WLAN devices have higher transmission power than ZigBee devices. Then, the high transmission power of WLAN devices causes the capture effect when WLAN and ZigBee transmit simultaneously. On the other hand, ZigBee performs backoff during clear channel assessment (CCA) operation if the WLAN is transmitting its frames on the channel. We adopt a widely used WLAN queueing/transmission model that is based on a Markov chain concept. We model a ZigBee device's operation using the Markov chain that includes WLAN interference statistically derived from the WLAN queueing/transmission model. Our proposed algorithm is evaluated in a simulated ZigBee network in the presence of varying WLAN interference. Numerical results show that our WLAN interference mitigation scheme finds the ZigBee control parameters, among a candidate set, which enhances ZigBee network performance compared to the conventional ZigBee operation.