State Markov Chain Research Articles

On Markov−switching asymmetric logGARCH models: Stationarity and estimation

In the present paper, we study some probabilistic and statistical properties of the Markov-switching asymmetric logGARCH processes, where the log?volatility follows a standard asymmetric logGARCH process for each regime. In these models, the coefficients of log?volatility depend on the state of a non-observed Markov chain. The main motivations of this new model can capture the asymmetries and hence leverage effect. Additionally, The volatility coefficients are not subject to positivity constraints. Therefore, some probabilistic properties of Markov-switching asymmetric logGARCH models have been obtained, especially, sufficient conditions ensuring the existence of stationary, causal, ergodic solution and moments properties are given. Furthermore, we show the strong consistency of the quasi-maximum likelihood estimator (QMLE) under mild assumptions. Finally, we provide a simulation study of the performance of the proposed estimation method and the MS?logGARCH is applied to model the exchange rate of the Algerian Dinar against the US-dollar.

Optimal consumption, investment, and insurance under state-dependent risk aversion

AbstractWe formalize a consumption–investment–insurance problem with the distinction of a state-dependent relative risk aversion. The state dependence refers to the state of the finite state Markov chain that also formalizes insurable risks such as health and lifetime uncertainty. We derive and analyze the implicit solution to the problem, compare it with special cases in the literature, and illustrate the range of results in a disability model where the relative risk aversion is preserved, decreases, or increases upon disability.

Stochastic modelling of the daily precipitation occurrence in the Tocantins-Araguaia hydrographic region, Amazon, Brazil

Evaluating the temporal and spatial variation of the occurrence of dry and rainy days is essential for planning the use of water resources. This article evaluated the behaviour of daily precipitation in the Tocantins-Araguaia hydrographic region (TAHR). The 1st-order stationary Markov chain (MC) of two states was used without considering the spatial correlation between the rainfall gauge stations. The developed model was efficient in estimating the occurrences of daily precipitation for the entire TAHR, as the maximum errors found were equal to RMSE: 14.1568, RSR: 0.0149, NSE: 0.9973 and PBIAS: -1.7650. However, the areas located in the Amazon-Cerrado Biome (Ecotone) require more attention due to the greater occurrence of consecutive drought days. Therefore, it is suitable to use MCs of higher orders for greater accuracy. In addition, the transition probabilities that led to rain conditions were greater when they were closer to the Atlantic Ocean, while the transition probabilities that led to drought conditions were greater in the areas located in the Cerrado biome.

Age of information analysis for internet of things information status update system with cellular backhaul and dedicated energy harvesting beacons

SummaryThe Internet of Things (IoT) that is usually deployed with the assistance of cellular backhaul and energy harvesting (EH) is characterized by the status update freshness. However, the traditional nature EH results in the loss of information freshness due to the randomness of nature energy process. Although using wireless energy transferring (WET) to charge sensors can overcome this problem, this method has low‐energy efficiency. With these considerations, this paper proposes a novel IoT system under cellular communication scenario by simultaneously exploiting the nature EH and WET. The proposed IoT system is consisted of a cellular backhaul subsystem, a wireless EH and transferring (WEHT) subsystem, and a wireless sensor status update subsystem. The WEHT subsystem employs the dedicated power beacons harvesting energy from natural energy source and transferring the harvested energy to sensor. For the proposed IoT system, this paper first investigates the WEHT subsystem and constructs the Markov Chain (MC) of discrete energy states as well as the MC state transition matrix. Second, the average AoI and peak AoI (PAoI) are formulated by separately considering cellular backhaul and EHTBs. The AoI (PAoI) comparison shows that the proposed EHTB‐based IoT system can outperform the conventional non‐EHTB one where the sensor directly harvests energy from natural source. At the same time, the numerical results exploit the impact of system parameters on AoI and PAoI, respectively. It is found that there exists a trade‐off between the nature energy arrival and EHTB energy transfer so that the average minimum AoI and PAoI are achieved.

EMI-Regulated SiC-Based Motor Drives With Markov Chain Pseudo-Random Frequency Space Vector Pulse Width Modulation Strategy

Compared with traditional silicon-based (Si) power devices, wide-bandgap (WBG) power devices have broader application prospects in AC motor drive due to their high-temperature reliability, low conduction, and switching losses. However, a fast switching process and high switching frequency will cause more serious conducted electromagnetic interference (EMI) emission. To solve the problem, this paper proposed a pseudo-random frequency space vector pulse width modulation (PRF-SVPWM) strategy based on a carrier with a pseudo-random frequency. There are two sawtooth waves with the same frequency but opposite phases as alternatives. The proposed method randomly selects only one sawtooth wave during a complete control period. After random combination, the generated carrier is no longer a regular sawtooth wave. This new carrier contains 4 different modes including 2 sawtooth (positive and negative) and 2 triangular waves (positive and negative). A 2-state Markov chain is introduced to produce a PRF control signal which is regarded as the law of choice. In this way, the equivalent switching frequency keeps constant but the real-time switching frequency will randomly vary within the determined range. EMI emission level can be significantly reduced. The effectiveness of the proposed PRF-SVPWM is verified by several sets of experiment results.

Outage Performance of Multi-relay System with Energy Harvesting and Storage

<p>This paper analyzes the performance of a two-hop half-duplex multi-relay system based on energy harvesting. The relay has energy harvesting and storage functions, and adopts an adaptive AF/DF transmission strategy and PS protocol. Based on three relay selection schemes, namely Energy Optimal Selection (EOS), Channel Gain Optimal Selection (GOS), and Energy and Channel Gain Optimal Selection (EGOS), the outage performance is compared. First, the finite state Markov chain (FSMC) is used to model the energy arrival and use status of each relay, and the energy transfer steady-state matrix is obtained. Then we use the Gauss Chebyshev formula to derive the analytical expression of the communication outage probability (COP). Finally, an optimization model for minimizing the COP for three relay selection schemes is constructed. The EOS and GOS schemes optimize power allocation for energy harvesting and information transmission, and the EGOS scheme jointly optimizes power allocation and energy thresholds. We use a one-dimensional search algorithm based on the golden section and a two-dimensional search algorithm based on iteration to solve these optimization problems. The simulation results show that the EGOS is the best among the three relay selection schemes.</p> <p> </p>

China's progress toward sustainable development in pursuit of carbon neutrality: Regional differences and dynamic evolution

China has proposed ambitious carbon neutrality targets to address the challenge of climate change and promote sustainable development despite being the world's largest emitter of carbon dioxide. This study constructs an index system with six dimensions: carbon source, carbon sink, electrification, low-carbon technology, the government in action, and economic decarbonization. Furthermore, it measures the low-carbon sustainable development of China's provinces as an assessment of the progress toward carbon neutrality. The comprehensive scores show that most provinces in China have progressed in their pursuit of carbon neutrality from 2007 to 2018. However, improvements can still be made. Low-carbon technology is the advantageous for low-carbon sustainable development in the eastern region and the driving force of the catch-up effect in the central region. In addition, this study finds that the overall differences in provincial carbon neutralization scores are widening. Specifically, inter-regional differences are leading, followed by intra-regional differences. Convergence analysis also shows that the carbon neutralization scores in different provinces exhibit a non-convergence trend or even an expanding trend. A large regional imbalance exists in low-carbon sustainable development. The kernel density estimation and Markov chain state analysis show that China's provinces are moving toward carbon neutrality, and the absolute difference in carbon neutrality scores is also widening. However, a phenomenon of club convergence exists.

Mine ventilation system reliability evaluation based on a Markov chain

Mine ventilation systems play a key role in creating and sustaining a healthy and safe working environment within the mine, and as such, should always be maintained at optimal performance levels. This paper establishes a model based on Markov chain that can quickly evaluate the reliability of the ventilation system. Firstly, the operation status of the ventilation system is divided into normal, risk and failure. Then, according to the failure rate and repair rate of the system, the operation state of the system under the specified total operation time T and time interval Δt is simulated based on Monte Carlo method, the Markov chain state transfer probability matrix of the system can be obtained. Combined with the current operation state of the system, the reliability indexes such as the system operation state transfer probability and the steady state probability in the future can be quickly analyzed to realize the rapid evaluation of the operation reliability of the ventilation system. Finally, the model is used to evaluate the reliability of XQ mine ventilation system, which shows the effectiveness of the model. This research provides theoretical reference and technical support for mine safety production.

Generating MCMC proposals by randomly rotating the regular simplex

We present the simplicial sampler, a class of parallel MCMC methods that generate and choose from multiple proposals at each iteration. The algorithm’s multiproposal randomly rotates a simplex connected to the current Markov chain state in a way that inherently preserves symmetry between proposals. As a result, the simplicial sampler leads to a simplified acceptance step: it simply chooses from among the simplex nodes with probability proportional to their target density values. We also investigate a multivariate Gaussian-based symmetric multiproposal mechanism and prove that it also enjoys the same simplified acceptance step. This insight leads to significant theoretical and practical speedups. While both algorithms enjoy natural parallelizability, we show that conventional implementations are sufficient to confer efficiency gains across an array of dimensions and a number of target distributions.

A Monitoring Method Based on FDALM and Its Application in the Sintering Process of Ternary Cathode Material.

In industrial processes, the composition of raw material and the production environment are complex and changeable, which makes the production process have multiple steady states. In this situation, it is difficult for the traditional single-mode monitoring methods to accurately detect the process abnormalities. To this end, a multimode monitoring method based on the factor dynamic autoregressive hidden variable model (FDALM) for industrial processes is proposed in this paper. First, an improved affine propagation clustering algorithm to learn the model modal factors is adopted, and the FDALM is constructed by combining multiple high-order hidden state Markov chains through the factor modeling technology. Secondly, a fusion algorithm based on Bayesian filtering, smoothing, and expectation-maximization is adopted to identify model parameters. The Lagrange multiplier formula is additionally constructed to update the factor coefficients by using the factor constraints in the solving. Moreover, the online Bayesian inference is adopted to fuse the information of different factor modes and obtain the fault posterior probability, which can improve the overall monitoring effect of the model. Finally, the proposed method is applied in the sintering process of ternary cathode material. The results show that the fault detection rate and false alarm rate of this method are improved obviously compared with the traditional methods.

On Adaptive Transmission for Distributed Detection in Energy Harvesting Wireless Sensor Networks With Limited Fusion Center Feedback

We consider a wireless sensor network tasked with solving a binary distributed detection problem. Sensors communicate directly with a fusion center (FC) over orthogonal fading channels, with additive Gaussian noise. Each sensor can harvest randomly arriving energy units and store them in a battery. Also, it knows its quantized channel state information (CSI), acquired via a limited feedback channel from the FC. Modeling the randomly arriving energy units during a time slot as a Poisson process and the battery dynamics as a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${K}$ </tex-math></inline-formula> -state Markov chain (where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${K}$ </tex-math></inline-formula> is the battery size), we propose a channel-dependent transmit power control strategy such that the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${J}$ </tex-math></inline-formula> -divergence based detection metric is maximized at the FC, subject to an average transmit power per sensor constraint. The proposed strategy is parametrized in terms of the channel gain quantization thresholds and the scale factors corresponding to the quantization intervals. This strategy allows each sensor to adapt its transmit power based on its battery state and its qunatized CSI. Finding optimal strategy requires solving a non-convex optimization problem that is not differentiable with respect to the optimization variables. We propose near-optimal strategies based on random search methods that have a low-computational complexity and provide a close-to-optimal performance.

Status update control based on reinforcement learning in energy harvesting sensor networks

With the development of the Internet of Things, more and more sensors are deployed to monitor the environmental status. To reduce deployment costs, a large number of sensors need to be deployed without a stable grid power supply. Therefore, on the one hand, the wireless sensors need to save as much energy as possible to extend their lifetime. On the other hand, they need to sense and transmit timely and accurate information for real-time monitoring. In this study, based on the spatiotemporal correlation of the environmental status monitored by the sensors, status information estimation is considered to effectively reduce the information collection frequency of the sensors, thereby reducing the energy cost. Under an ideal communication model with unlimited and perfect channels, a status update scheduling mechanism based on a Q-learning algorithm is proposed. With a nonideal channel model, a status update scheduling mechanism based on deep reinforcement learning is proposed. In this scenario, all sensors share a limited number of channels, and channel fading is considered. A finite state Markov chain is adopted to model the channel state transition process. The simulation results based on a real dataset show that compared with several baseline methods, the proposed mechanisms can well balance the energy cost and information errors and significantly reduce the update frequency while ensuring information accuracy.

Markov Chain Monte Carlo for generating ranked textual data

This paper faces a central theme in applied statistics and information science, which is the assessment of the stochastic structure of rank-size laws in text analysis. We consider the words in a corpus by ranking them on the basis of their frequencies in descending order. The starting point is that the ranked data generated in linguistic contexts can be viewed as the realisations of a discrete states Markov chain, whose stationary distribution behaves according to a discretisation of the best fitted rank-size law. The employed methodological toolkit is Markov Chain Monte Carlo, specifically referring to the Metropolis–Hastings algorithm. The theoretical framework is applied to the rank-size analysis of the hapax legomena occurring in the speeches of the US Presidents. We offer a large number of statistical tests leading to the consistency of our methodological proposal. To pursue our scopes, we also offer arguments supporting that hapaxes are rare (“extreme”) events resulting from memory-less-like processes. Moreover, we show that the considered sample has the stochastic structure of a Markov chain of order one. Importantly, we discuss the versatility of the method, which is considered suitable for deducing similar outcomes for other applied science contexts.

Generating Synthetic Monthly Net Radiation Data Conditioned on Wind Speed

Aims: This study aimed at generating synthetic monthly net radiation data conditioned on wind speed over Port Harcourt, Benue, Kano and Enugu in Nigeria.&#x0D; Duration of Study: The daily maximum and minimum Relative-Humidity, maximum and minimum air temperature, solar radiation and wind speed data was obtained from the International Institute of Tropical Agriculture (IITA) Ibadan, Nigeria for the period of thirty-four (34) years (1977-2010).&#x0D; Method: The Penman-Monteith (FAO-56) step by step method was used to compute the daily net radiation. The autocorrelation function was used to establish the fact that net radiation and wind speed exhibit the Markov property. Lastly, a two – state Markov Chain model and an indicator function of energy balance and imbalance was developed and used in the course of this work.&#x0D; Results: The finding reveals that net radiation is surplus over the study areas. The Lag one autocorrelation coefficient confirms the fact that an actual day net radiation and wind speed state depends on the previous day’s state. The monthly steady state probabilities of surplus net radiation conditioned on low wind speed is higher compared to monthly steady state probabilities of surplus net radiation conditioned on high wind speed. The indicator function of energy balance or imbalance reveals that there is energy imbalance over study areas.&#x0D; Conclusion: The generated synthetic net radiation data conditioned on wind speed preserved the characteristics of actual net radiation data when compared as observed in the study. This data is essential in the study of climate change, weather monitoring, agricultural meteorology and estimation of evapotranspiration.

On the choice of number of superstates in the aggregation of Markov chains

• Propose a notion of Marginal Return to compare aggregated models of a Markov chain. • Above comparison facilitates estimating the appropriate size for the aggregated chain. • Proposed method extends to several model-reduction problems such as co-clustering. • Simulations demonstrate efficacy on a range of synthetic and real datasets. Many studies involving large Markov chains require determining a smaller representative (aggregated) chain. Each superstate in the representative chain represents a group of related states in the original Markov chain. Typically, the choice of number of superstates in the aggregated chain is ambiguous, and based on the limited prior know-how. This paper presents a structured methodology of determining the best candidate for the number of superstates. It achieves this by comparing aggregated chains of different sizes. To facilitate this comparison a new quantity called heterogeneity of a superstate is developed, and subsequently it is used to establish the notion of marginal return of an aggregated chain. In particular, the notion of marginal return captures the decrease in the heterogeneity upon a unit increase in the number of superstates in the aggregated chain. Maximum Entropy Principle (MEP), from statistical mechanics, justifies the developed notion of marginal return, as well as the quantification of heterogeneity. Through simulations on synthetic Markov chains, where the number of superstates are known a priori, it is observed that the aggregated chain with the largest marginal return identifies this number. In case of Markov chains that model real-life scenarios it is shown that the aggregated model with the largest marginal return identifies an inherent structure unique to the scenario being modelled; thus, substantiating the efficacy of the proposed methodology.

Some Old and New Problems in Thermodynamic Formalism of Infinite State Markov Chains

Some Old and New Problems in Thermodynamic Formalism of Infinite State Markov Chains

HTA16 An MCDA Approach to Identify Direct Relationships Between Health States in Markov Chains for Heart Failure Treatments

To identify direct relationships between health states in two heart failure treatments Markov chains. After that, this study seeks to observe if the identified direct relationships between health states for the studied treatments are the same.

Multi-sector discrete-time channel model for data link layer evaluation of CubeSat communications

Innovation in the space sector is going through a fruitful period. Among other factors, this has been a consequence of the CubeSat paradigm and its acceptance in the educational, scientific, and commercial sectors, as well as its relevant role in the integration of terrestrial and satellite communication systems. Nevertheless, there is no standardized communication protocol for CubeSat technology. One of the reasons for this lack of standardization is the nonexistence of a widely accepted communication channel model for fast development and simulation of CubeSat technology at the data link layer level. Aiming at filling this gap, we propose a communication channel model that captures both the dynamics of the CubeSat in the low earth orbit and the propagation environment where the terrestrial devices are located, taking into account the elevation angle. For this purpose, finite state Markov chains are used to model the time evolution of the channel fading through the satellite trajectory when it passes by the ground point target. Then, CCSDS recommendations are used as a study case for the data link layer of a CubeSat, while an adaptive automatic repeat request mechanism, according to the elevation angle, is simulated and optimized, showing the usefulness of the proposed channel model.

Probabilistic Analysis of Covid-19 Pandemic in Kenya Using Markov Chain

Since the inception of Covid-19 in China, the economies around the world have been on the turmoil. This is because China has a direct correlation with most economies in the world; they depend on it directly or indirectly. On 13th March, 2020 the first case of COVID-19 in Kenya a 27-year-old Kenyan woman who traveled from the US via London, was confirmed. The Kenyan government identified and isolated a number of people who had come into contact with the first case. On 15 March 2020, the president of Kenya directed that a number of measures be taken to curb COVID-19, some of those measures included; dusk to dawn curfew, secession of movement and mandatory quarantine of suspected cases. Based on the available literature, probabilistic predictions using steady state Markov chain allow to assess the uncertainty of the COVID-19 comprehensively. Therefore they are preferable to forecasts for the mean or median COVID-19 only. The probabilistic COVID-19 predictions allow to derive probabilistic forecasts for the number of patients who are still at the ICU at a certain day in future. This may be useful for planning purposes. From the probabilities for single patients, one may compute the probability that any given number of patients is still at the ICU after t days. However, in Kenya there is scanty information on analysis of COVID-19 using steady state Markov Chain. The aim of this study was therefore be to carry out probabilistic analysis of COVID-19 pandemic in Kenya using Markov chain. The study was a literature based, in which the researcher reviewed surveys books, scholarly journals, and other secondary sources relevant to the current study topic. The findings revealed that one of the most important uses of steady state Markov chain in analyzing COVID-19 pandemic situation in Kenya is that it compares performances for different states of affairs and courses of action within the health sector, by using system steady state performance measurements. The study concludes that steady state Markov chain is beneficial in simulating the corona infection in numerous stages. It is thus recommended that there is need for policy-makers to seek regional and global solutions to COVID-19 disease instead of limited solutions within the country. Keywords: Steady State, Markov chain, COVID-19 Pandemic, Transition Matrix

Predicting the Evolution of Controlled Systems Modeled by Finite Markov Processes

The operation and maintenance of infrastructure components and systems can be modeled as a Markov process, partially or fully observable. Information about the current condition can be summarized by the &#x201C;inner&#x201D; state of a finite state controller. When a control policy is assigned, the stochastic evolution of the system is completely described by a Markov transition function. This article applies finite state Markov chain analyses to identify relevant features of the time evolution of a controlled system. We focus on assessing if some critical conditions are reachable (or if some actions will ever be taken), in identifying the probability of these critical events occurring within a time period, their expected time of occurrence, their long-term frequency, and the probability that some events occur before others. We present analytical methods based on linear algebra to address these questions, discuss their computational complexity and the structure of the solution. The analyses can be performed after a policy is selected for a Markov decision process (MDP) or a partially observable MDP. Their outcomes depend on the selected policy and examining these outcomes can provide the decision makers with deeper understanding of the consequences of following that policy, and may also suggest revising it.