Markov Chain Model Research Articles

ECG Modeling for Simulation of Arrhythmias in Time-Varying Conditions.

The present article proposes an ECG simulator that advances modeling of arrhythmias and noise by introducing time-varying signal characteristics. The simulator is built around a discrete-time Markov chain model for simulating atrial and ventricular arrhythmias of particular relevance when analyzing atrial fibrillation (AF). Each state is associated with statistical information on episode duration and heartbeat characteristics. Statistical, time-varying modeling of muscle noise, motion artifacts, and the influence of respiration is introduced to increase the complexity of simulated ECGs, making the simulator well suited for data augmentation in machine learning. Modeling of how the PQ and QT intervals depend on heart rate is also introduced. The realism of simulated ECGs is assessed by three experienced doctors, showing that simulated ECGs are difficult to distinguish from real ECGs. Simulator usefulness is illustrated in terms of AF detection performance when either simulated or real ECGs are used to train a neural network for signal quality control. The results show that both types of training lead to similar performance.

Spatial and temporal evolution of the eco-efficiency of cultivated land use in the region around Beijing–Tianjin based on the Super-EBM model

The eco-efficiency of cultivated land use (ECLU) is an important indicator for ecological civilization construction in China. Exploring the spatiotemporal dynamic evolution of the ECLU is helpful for the sustainable use of arable land, ensuring food security and ecological security. However, previous studies have mostly focused on the use of a slacks-based measure (SBM) model for ECLU measurement, ignoring the more accurate epsilon-based measure (EBM) model. Therefore, in this study, first, we explored the conceptual framework of the ECLU, and then, based on 2005–2020 panel data covering the counties in the region around Beijing and Tianjin, we investigated the spatial and temporal evolution of the ECLU by using the Super-EBM model, kernel density estimation method, and spatial Markov chain model. The results show the following: 1) From 2005 to 2020, the ECLU in the region around Beijing and Tianjin displayed an increasing state, but the average value was only 0.55. 2) The temporal evolution of the ECLU has gradually polarized, and the internal gap has widened, but it tends to stabilize. 3) The ECLU in the region around Beijing and Tianjin was more inclined to remain the same, and there was a “club convergence” phenomenon, which was meaningfully affected by the background of neighboring areas. In this article, we emphasize the significance of the super‐efficient EBM model in measuring the ecological efficiency of arable land use, and discuss the spatiotemporal evolution of the ecological efficiency of arable land use at a deeper level. In light of local conditions, the government should reasonably formulate a path to optimize the ECLU, strengthen the linkage with surrounding cities, and bring into play the positive spillover effect.

Filtering in Triplet Markov Chain Model in the Presence of Non-Gaussian Noise with Application to Target Tracking

In hidden Markov chain (HMC) models, widely used for target tracking, the process noise and measurement noise are in general assumed to be independent and Gaussian for mathematical simplicity. However, the independence and Gaussian assumptions do not always hold in practice. For instance, in a typical radar tracking application, the measurement noise is correlated over time as the sampling frequency of a radar is generally much higher than the bandwidth of the measurement noise. In addition, target maneuvers and measurement outliers imply that the process noise and measurement noise are non-Gaussian. To solve this problem, we resort to triplet Markov chain (TMC) models to describe stochastic systems with correlated noise and derive a new filter under the maximum correntropy criterion to deal with non-Gaussian noise. By stacking the state vector, measurement vector, and auxiliary vector into a triplet state vector, the TMC model can capture the complete dynamics of stochastic systems, which may be subjected to potential parameter uncertainty, non-stationarity, or error sources. Correntropy is used to measure the similarity of two random variables; unlike the commonly used minimum mean square error criterion, which uses only second-order statistics, correntropy uses second-order and higher-order information, and is more suitable for systems in the presence of non-Gaussian noise, particularly some heavy-tailed noise disturbances. Furthermore, to reduce the influence of round-off errors, a square-root implementation of the new filter is provided using QR decomposition. Instead of the full covariance matrices, corresponding Cholesky factors are recursively calculated in the square-root filtering algorithm. This is more numerically stable for ill-conditioned problems compared to the conventional filter. Finally, the effectiveness of the proposed algorithms is illustrated via three numerical examples.

Characterization of sleep efficiency transitions in family caregivers.

Family caregivers are at high risk of psychological distress and low sleep efficiency resulting from their caregiving responsibilities. Although psychological symptoms are associated with sleep efficiency, there is limited knowledge about the association of psychological distress with variations in sleep efficiency. We aimed to characterize the short- and long-term patterns of caregivers' sleep efficiency using Markov chain models and compare these patterns between groups with high and low psychological symptoms (i.e., depression, anxiety, and caregiving stress). Based on 7-day actigraphy data from 33 caregivers, we categorized sleep efficiency into threestates, < 75% (S1), 75-84% (S2), and ≥ 85% (S3), and developed Markov chain models. Caregivers were likely to maintain a consistent sleep efficiency state from one night to the next without returning efficiently to a normal state. On average, it took 3.6-5.1days to return to a night of normal sleep efficiency (S3) from lower states, and the long-term probability of achieving normal sleep was 42%. We observed lower probabilities of transitioning to or remaining in a normal sleep efficiency state (S3) in the high depression and anxiety groups compared to the low symptom groups. The differences in the time required to return to a normal state were inconsistent by symptom levels. The long-term probability of achieving normal sleep efficiency was significantly lower for caregivers with high depression and anxiety compared to the low symptom groups. Caregivers' sleep efficiency appears to remain relatively consistent over time and does not show rapid recovery. Caregivers with higher levels of depression and anxiety may be more vulnerable to sustained suboptimal sleep efficiency.

Achieving Value by Risk Stratification With Machine Learning Model or Clinical Risk Score in Acute Upper Gastrointestinal Bleeding: A Cost Minimization Analysis.

We estimate the economic impact of applying risk assessment tools to identify very low-risk patients with upper gastrointestinal bleeding who can be safely discharged from the emergency department using a cost minimization analysis. We compare triage strategies (Glasgow-Blatchford score = 0/0-1 or validated machine learning model) with usual care using a Markov chain model from a US health care payer perspective. Over 5 years, the Glasgow-Blatchford score triage strategy produced national cumulative savings over usual care of more than $2.7 billion and the machine learning strategy of more than $3.4 billion. Implementing risk assessment models for upper gastrointestinal bleeding reduces costs, thereby increasing value.

A Markov chain model of crop conditions and intrayear crop yield forecasting

AbstractCrop condition reports are an important source of information for producers, grain traders, businesses, and policymakers to assess and manage the price and yield risk inherent in a given crop. A Markov chain model is proposed for describing the weekly dynamic behavior of reported crop conditions. Empirical transition probabilities are estimated for corn grown in Nebraska, and forecasted crop conditions from the Markov chain are used as inputs to forecast final crop yields prior to harvest time. The results suggest that the modeling and forecasting approach has value for estimating crop yields as intrayear information about crop conditions materializes.

Enhancing the effectiveness of bioaerosol disinfection in indoor environments by optimizing far-UVC lamp locations based on Markov chain model

Far-ultraviolet C (far-UVC) light is an effective and safe disinfection method for bioaerosol control in occupied indoor environments. The installation location of a far-UVC lamp strongly influences the spatial distribution of far-UVC irradiance, and thus the effectiveness of bioaerosol disinfection. To assist the design process, this study developed a fast prediction approach based on the Markov chain model for optimizing the installation locations of far-UVC lamps in order to enhance the disinfection effectiveness for indoor bioaerosol control. Experiments were conducted in an environmental chamber to validate the proposed simulation-based optimization approach. The results show that the proposed method can correctly predict the disinfection efficiency when compared with experimental data, and optimizing the installation location of the far-UVC lamp increased the disinfection efficiency by 54 % compared with the worst location. As an application, the validated method was then used to design the installation location of a far-UVC lamp in a real conference room. The results show that installing the far-UVC lamp in the optimal location can increase the disinfection efficiency by 48 % compared with the worst installation location. Therefore, optimizing the far-UVC lamp location using the proposed Markov chain model can enhance the effectiveness of bioaerosol disinfection in indoor environments.

Pairwise and higher-order epistatic effects among somatic cancer mutations across oncogenesis

Cancer occurs as a consequence of multiple somatic mutations that lead to uncontrolled cell growth. Mutual exclusivity and co-occurrence of mutations imply—but do not prove—that mutations exert synergistic or antagonistic epistatic effects on oncogenesis. Knowledge of these interactions, and the consequent trajectories of mutation and selection that lead to cancer has been a longstanding goal within the cancer research community. Recent research has revealed mutation rates and scaled selection coefficients for specific recurrent variants across many cancer types. However, there are no current methods to quantify the strength of selection incorporating pairwise and higher-order epistatic effects on selection within the trajectory of likely cancer genotoypes. Therefore, we have developed a continuous-time Markov chain model that enables the estimation of mutation origination and fixation (flux), dependent on somatic cancer genotype. Coupling this continuous-time Markov chain model with a deconvolution approach provides estimates of underlying mutation rates and selection across the trajectory of oncogenesis. We demonstrate computation of fluxes and selection coefficients in a somatic evolutionary model for the four most frequently variant driver genes (TP53, LRP1B, KRAS and STK11) from 565 cases of lung adenocarcinoma. Our analysis reveals multiple antagonistic epistatic effects that reduce the possible routes of oncogenesis, and inform cancer research regarding viable trajectories of somatic evolution whose progression could be forestalled by precision medicine. Synergistic epistatic effects are also identified, most notably in the somatic genotype TP53 LRP1B for mutations in the KRAS gene, and in somatic genotypes containing KRAS or TP53 mutations for mutations in the STK11 gene. Large positive fluxes of KRAS variants were driven by large selection coefficients, whereas the flux toward LRP1B mutations was substantially aided by a large mutation rate for this gene. The approach enables inference of the most likely routes of site-specific variant evolution and estimation of the strength of selection operating on each step along the route, a key component of what we need to know to develop and implement personalized cancer therapies.

A landscape ecology approach: Modeling forest fragmentation with artificial neural networks and cellular‐automata Markov‐chain for improved environmental policy in the southwestern Brazilian Amazon

AbstractLand degradation and forest fragmentation have been prominent issues in the Amazon since the 1970s, primarily driven by the suppression of primary forests due to land use changes. In this study, we propose an innovative approach by integrating artificial neural networks (ANN) and cellular automata Markov chain (CA‐MA) models to predict future land use and assess the associated forest fragmentation using landscape ecology metrics in the Jamari River Basin. The analysis reveals a significant increase in fragmentation between 1985 and 2018, as evidenced by a rise in the number of fragments from 7162 to 28,170. Moreover, we observed a decrease in the core area from approximately 2.5 million hectares to less than 1 million hectares, accompanied by an increase in edge density from 6 to 18 m.ha−1. Additionally, the average distance between fragments expanded from 66 to 95 m. Highlighting the urgency of addressing this issue, our study emphasizes the necessity of exploring effective strategies within the Brazilian public environmental management system. By utilizing various rural policy planning tools, we aim to tackle unsustainable development and mitigate the adverse impacts of land use changes in the region. In conclusion, this research offers an innovative approach that combines ANN and CA‐MA models to predict land use and assess forest fragmentation, shedding light on the alarming trend of land degradation and providing valuable insights for informed decision‐making and sustainable land management practices.

Research on the application of Markov chain

In modern times, mathematicians are often troubled by new approaches to deducing the outcomes of certain events. The introduction of Markov models eliminated queries across different sectors. In 1907, Russian mathematician Andrey Markov proposed the concept of Markov chains. It has been widely used in many aspects, such as weather prediction, deep learning, biological information, and so on. Therefore, this paper examines how the Markov chain model can be applied in a variety of situations. This study uses Python as a supporting tool to simulate states and possible outcomes. It can be concluded that Python is able to simulate the state transitions of the Markov chain. The paper also identifies the differences between Markov models, their application in common scenarios such as medical, finance, weather forecasting, machine learning and others in our everyday life and why they are so popularly used, including the simplicity of the model and more.

The Predictability of Stock Price: Empirical Study on Tick Data in Chinese Stock Market

Whether or not stocks are predictable has been a topic of concern for decades. The efficient market hypothesis (EMH) says that it is difficult for investors to make extra profits by predicting stock prices, but this may not be true, especially for the Chinese stock market. Therefore, we explore the predictability of the Chinese stock market based on tick data, a widely studied high-frequency data. We obtain the predictability of 3, 834 Chinese stocks by adopting the concept of true entropy, which is calculated by Limpel-Ziv data compression method. The Markov chain model and the diffusion kernel model are used to compare the upper bounds on predictability, and it is concluded that there is still a significant performance gap between the forecasting models used and the theoretical upper bounds. Our work shows that more than 73% of stocks have prediction accuracy greater than 70% and RMSE less than 2 CNY under different quantification intervals with different models. We further take Spearman's correlation to reveal that the average stock price and price volatility may have a negative impact on prediction accuracy, which may be helpful for stock investors.

Generating and understanding human daily activity sequences using Time-Varying Markov Chain models

Human activity patterns have raised broad interests across fields and activity-based modeling has been applied to analyze travel behaviors, effects of policies, etc. In this paper, we examined daily activity sequences from large volumes of datasets collected in the Tokyo Metropolitan Area over thirty years. We show that, using a simple time-varying (inhomogeneous) Markov Chain model, activity sequences at the individual level can be generated with high accuracy. The predictive power of the model is comparable to neural network models when generating activity sequences for each administrative region. The results are surprising considering the huge number of potential activity sequences and the complicated process of activity planning. A fundamental relationship between time-varying Markov Chain models and Multinomial Logit models is recognized to understand the unexpected accuracy. A new mechanism is proposed on how people make daily activity plans from the perspective of cost and utility, where the probability of one plan being chosen is the same as the probability of it being generated by the Markov Chain model. From this perspective, the Markov process becomes a utility-based model. Furthermore, given the average time-varying transition probabilities over a large spatial and temporal scale, the Markov Chain model for any specific area is reconstructed with six parameters. The generation accuracy using the reconstructed model is examined. Moreover, the statistical characteristics of the costs assigned to the Markov Chain model are investigated.

Efficiency of agricultural modernization in China: Systematic analysis in the new framework of multidimensional security

Agricultural modernization is the foundation and pillar of national modernization, and its security should not be ignored. However, the traditional efficiency assessment does not take security factors into account. To improve this work, we tried to embed multidimensional security issues into the assessment framework of agricultural modernization efficiency (AME) and analyzed its efficiency levels, regional differences, and dynamic evolution using the Super-EBM model, Dagum Gini coefficient, and spatial Markov chain model. We found that: (1) At the national level, the agricultural operation efficiency (AOE) and agricultural service efficiency (ASE) were generally lower than the agricultural production efficiency (APE), with significant potential for improvement. The spatial characteristics exhibited evident spatial clustering, and the spatial differences in AME, AOE, and ASE were similar. (2) The overall Gini coefficient of AME fluctuates in the range of 0.1168–0.1334 from 2006 to 2020, and the differences in AME were dominated by inter-area differences, with an average contribution rate of 68.72%. (3) The spatial pattern of AME was relatively stable, with prominent “club convergence” characteristics, and spatial factors had adverse spillover effects on the evolution toward convergence. Policymakers should strive to fill the shortcomings of agricultural operations and agricultural services, formulate differentiated agricultural strategies for development and promote knowledge exchange and technological diffusion. These are important for building a modern agricultural system and promoting the spatially balanced development of modernized agriculture.

Inheritance of Chinese Traditional Music Based on Markov Chain Modelling

Abstract In this paper, we first propose a Markov chain-based automatic composition method to construct a song set using classical and traditional music as a sample. Using probability matrices, experiments are conducted to generate pitch and time value sequences, and melodic segments of traditional music are generated using transformation techniques. Secondly, a traditional music digital resource library that integrates multiple functions is constructed by utilizing various information technologies. Finally, based on the existing classification dimensions, the relationship between various types of influencing factors in the inheritance of traditional music was analyzed, and the structure of the questionnaire was initially constructed. The results show that the correlation coefficients between the cultural inheritance of traditional music and its resources, environmental threats, development level, diversified values, public demand, and social cognition are 0.4235, 0.0953, 0.3594, 0.4253, 0.5348, and 0.5224, respectively, and there is a significant positive influence. This paper provides some theoretical and practical guidance for the protection and inheritance of traditional music.

Biodiversity survey and estimation for line-transect sampling.

Conducting biodiversity surveys using a fully randomised design can be difficult due to budgetary constraints (e.g., the cost of labour), site accessibility, and other constraints. To this end, ecologists usually select representative line transects or quadrats from a studied area to collect individuals of a given species and use this information to estimate the levels of biodiversity over an entire region. However, commonly used biodiversity estimators such as Rao's quadratic diversity index (and especially the Gini-Simpson index) were developed based on the assumption of independent sampling of individuals. Therefore, their performance can be compromised or even misleading when applied to species abundance datasets that are collected from non-independent sampling. In this study, we utilise a Markov chain model and derive an associated parameter estimator to account for non-independence in sequential sampling. Empirical tests on two forest plots in tropical (Barro Colorado, Island of Panama) and subtropical (Heishiding Nature Reserve of Guangdong, China) regions and the continental-scale spatial distribution of Acacia species in Australia showed that our estimators performed reasonably well. The estimated parameter measuring the degree of non-independence of subsequent sampling showed that a non-independent effect is very likely to occur when using line transects to sample organisms in subtropical regions at both local and regional spatial scales. In summary, based on a first-order Markov sampling model and using Rao's quadratic diversity index as an example, our study provides an improvement in diversity estimation while simultaneously accounting for the non-independence of sampling in field biodiversity surveys. Our study presents one possible solution for addressing the non-independent sampling of individuals in biodiversity surveys.

A truncated test scheme design method for success-failure in-orbit tests

Based on the success-failure test feature of in-orbit tests (IOTs) for typical space equipment, this paper presents a method for designing a truncated test scheme for success-failure in-orbit tests. With this method, a small upper boundary of the sample size for the IOT verification test can be obtained before the test starts. The method introduces the truncated Bayes-sequential mesh test (SMT) method into the design of the IOT verification test scheme and greatly compresses the continuous test area by incorporating optimization theory, resulting in a smaller upper limit of the IOT sample size. First, this paper derives a specific calculation formula for the Bayes-SMT critical line. Second, the Markov chain model is adopted to calculate the occurrence probabilities of each acceptance and rejection point through state transition. Finally, an optimal truncated test optimization algorithm based on the augmented lagrangian genetic algorithm is proposed. Simulation tests show that, compared with the classical single sampling method, the truncated sequential probability ratio test method, the truncated SMT method, and the truncated Bayes-SMT method based on step-by-step calculation, the method presented in this paper can be used to obtain a sequential test scheme with smaller truncated sample size.

Modeling of emergency evacuation in high rise buildings considering congestion at stairs based on Markov chains

High-rise buildings are vulnerable to encountering fires or other emergencies at any time, requiring an immediate evacuation of occupants. In the context of building evacuations, various engineering parameters are often utilized to evaluate the time required for people to reach a safe place when descending a staircase. A discrete-time Markov chain model is proposed to evaluate the evacuation time of the occupants in such circumstances by taking into consideration factors like merging at the stairs and congestion on the floor. Based on the congestion probabilities with respect to the number of evacuees in the building and other human behavior factors that slow down the rate of evacuation, non-congested and congested scenarios are evaluated. The effect of the critical parameters that influenced the evacuation process in the high-rise building, such as walking speed, number of evacuees, and effective stair width, is evaluated in both scenarios. The findings indicate an optimal evacuation speed of 0.7–1.4 m/s. The effective width of the stair is suggested to be between 1.6 and 2.4 m using the model. The simulation results are in good accordance with the SFPE method, the NIST egress simulator, and the agent-based model results.

Outage Analysis of IRS-Assisted RF Powered Networks for Energy-Constrained IoT Devices

This paper studies the performance of an intelligent reflecting surface (IRS)-assisted radio frequency (RF)-powered wireless network. In the considered network, a multi-antenna base station (BS) communicates with its users (UEs) through downlink transmission and energy-constrained internet-of-things (IoT) devices, which are randomly deployed around a cluster head (CH), utilize the resources of BS for energy harvesting and information transmission. IRS, deployed randomly around CH, reflects the RF signals transmitted by BS to the IoT devices, providing additional energy to harvest. For the considered network, average distances between BS, CH, IRS, and an IoT device have been computed, statistically. Furthermore, the average harvested energy at an IoT device is computed. Instead of considering a fixed transmit power (FTP) of an IoT device, we consider that its transmit power varies randomly depending on the stored energy. Using the Markov chain model, we obtain transmission probability and state-dependent transmit power (SDTP). Analytical expressions for outage probability and throughput of an IoT device are obtained. To manifest the effectiveness of derived expressions, numerical results are compared with Monte-Carlo simulation results. Our results reveal that the proposed SDTP scheme has a performance gain over the FTP scheme, especially in scenarios where harvested energy is in abundance.

Innovative development of music education model in universities under stochastic process model analysis

Abstract Based on the analysis of random phenomena, random variables, and random processes, this paper constructs a Markov chain-based model for evaluating the effectiveness of music teaching using Markov chains. Aiming at the core literacy of college music education, a new education model is proposed to cultivate students’ comprehensive music literacy and transform the method of college music education. Traditional evaluation methods and the Markov chain model are used to analyze data examples for the new model of music education. The probability of transferring the number of students whose first score was on the 89.9-80 scale to the 100-90 scale was 0.5115, greater than the probability value of 0.3650 for those still on the 89.9-80 scale. The transfer matrix of the experimental class was shifted to the left as a whole, indicating a high degree of student progress.

Robust Packetized MPC for Networked Systems Subject to Packet Dropouts and Input Saturation With Quantized Feedback.

This article develops a robust packetized predictive control framework to deal with the quantized-feedback control problem of networked systems subject to Markovian packet dropouts and input saturation. In the proposed framework, the Markov chain model of packet dropout is established from the link of the controller to the actuator. To deal with the quantized measurements, a robust packetized predictive control method is presented with a quantized-feedback law. The problem of unreliable transmission is addressed by proposing a packet dropout compensation strategy with a forgetting factor. An augmented Markovian jump system model is established to take the packet dropouts into account. The synthesis of packetized predictive control is then developed by minimizing a worst case cost function with respect to the model uncertainties. The recursive feasibility of the proposed controller design problem and the mean-square stability of the closed-loop systems are proved, respectively. The proposed packetized predictive control method is demonstrated by simulating a four-tank process system.