First-order Markov Research Articles

Duals of multiplicative relationships involving beta and gamma random variables

By interpreting four known multiplicative relationships involving independent beta and gamma random variables as scale mixtures, their ‘dual’ mixture relationships are obtained, making links to a variety of other distributional relationships. One consequence of these dual relationships is the provision of a number of first-order Markov processes with beta and gamma marginals.

Gated Attention Recurrent Neural Network: A Deeping Learning Approach for Radar-Based Precipitation Nowcasting

Precipitation nowcasting predicts the future rainfall intensity in local areas in a brief time that impacts directly on human life. In this paper, we express the precipitation nowcasting as a spatiotemporal sequence prediction problem. Predictive learning for a spatiotemporal sequence aims to construct a model of natural spatiotemporal processes to predict the future frames based on historical frames. The spatiotemporal process is an abstraction of some of the spatial things in nature that change with time, and they usually do not change very dramatically. To simplify the model and facilitate the training, we considered that the spatiotemporal process satisfies the generalized Markov properties. The natural spatiotemporal processes are nonlinear and non-stationary in many aspects. The processes are not satisfied with the first-order Markov properties when making predictions, such as the nonlinear movement, expansion, dissipation, and intensity enhancement of echoes. To describe such complex spatiotemporal variations, higher-order Markov models need to be used for the modeling. However, many of the previous models for spatiotemporal prediction constructed were based on first-order Markov properties, losing information on the higher-order variations. Thus, we propose a recurrent neural network which satisfies the multi-order Markov properties to create more accurate spatiotemporal predictions. In this network, the core component is the memory cell structure of the gated attention mechanism, which combines the current input information, extracts the historical state that best matches the existing input from the historical multi-period memory information, and then predicts the future. Through this principle of the gated attention, we could extract the historical state information that is richer and deeper to predict the future and more accurately describe the changing characteristics of motion. The experiments show that our GARNN network captures the spatiotemporal characteristics better and obtains excellent results in the precipitation forecasting with radar echoes.

Would I Lie to You? Using Markov Chains to Detect Careless Responding in Survey Data

Although careless response behaviors by survey participants bear considerable threats to the quality of the data, scholars often do not sufficiently address this issue in their analyses. This might be partially because existing approaches to detect careless responding are specialized to detect a certain type of careless response patterns, but fail to detect others. We develop a post-hoc measure, the Lazy Respondents (Laz.R) index, that is based on first-order Markov chains and the idea that careless respondents’ last response can be used to predict their next response. Using large, publicly available datasets to compare our Laz.R index to the most common precautionary and post-hoc measures, we find that (1) the Laz.R index correctly identifies patterns of careless respondents that some or all of the other indices overlook and (2) the Laz.R index avoids false positives (i.e., patterns that are flagged by single other indices, but which seem to capture truthful answering behavior). Furthermore, we suggest an approach to find a sample-specific cutoff value for careless respondents via kneedle algorithm. Finally, we introduce an R Shiny application that provides scholars a simple and readily available means to implement the Laz.R index.

Memory Channel Models of a Hybrid PLC-VLC Link for a Smart Underground Mine

In this article, discrete memory channel models of a low-cost, visible light communication (VLC) and a hybrid amplify-and-forward (AF) powerline communication (PLC) and VLC channel are derived using data obtained from an underground mining testbed. The testbed is set up close to the mining face, where blasting occurs. Then, the noise distributions as a result of the connection of a blasting equipment and the mains switchboard supply on the powerline channel are modeled. The baseband signals are communicated over software-defined radios, using multicarrier modulation in the VLC and hybrid AF PLC-VLC channels. Aided by a Fritchman 3-state semi-hidden Markov model (SHMM) for channel state classification, the channels are modeled using a first-order (FO) Markov process, which considers only one previous state in the channel. In addition, the block diagonal (BD) Markov model, which groups similar successive states of the channel, is implemented in order to reduce the training complexity. The models for the error distribution are then derived for both VLC and hybrid PLC-VLC channels. Results show that the probability of error obtained from the trained models closely matches the probability of error obtained from the channel measurements, validating the suitability of the SHMM for modeling VLC and hybrid PLC-VLC channels. In addition, the modified BD Markov model, which groups the channel states shows better performance with reduced complexity when compared with the FO Markov model.

Reward prediction errors, not sensory prediction errors, play a major role in model selection in human reinforcement learning

Model-based reinforcement learning enables an agent to learn in variable environments and tasks by optimizing its actions based on the predicted states and outcomes. This mechanism has also been considered in the brain. However, exactly how the brain selects an appropriate model for confronting environments has remained unclear. Here, we investigated the model selection algorithm in the human brain during a reinforcement learning task. One primary theory of model selection in the brain is based on sensory prediction errors. Here, we compared this theory with an alternative possibility of internal model selection with reward prediction errors. To compare these two theories, we devised a switching experiment from a first-order Markov decision process to a second-order Markov decision process that provides either reward- or sensory prediction error regarding environmental change. We tested two representative computational models driven by different prediction errors. One is the sensory prediction-error-driven Bayesian algorithm, which has been discussed as a representative internal model selection algorithm in the animal reinforcement learning task. The other is the reward-prediction-error-driven policy gradient algorithm. We compared the simulation results of these two computational models with human reinforcement learning behaviors. The model fitting result supports that the policy gradient algorithm is preferable to the Bayesian algorithm. This suggests that the human brain employs the reward prediction error to select an appropriate internal model in the reinforcement learning task.

Evaluation of low-cost formaldehyde sensors calibration

Low-cost sensors (LCS) are becoming ubiquitous in the market; however, calibration is needed before reliable use. An evaluation of the calibration of eight identical pre-calibrated formaldehyde LCS is presented here. The LCS and a reference instrument were exposed to a pollutant source(s) for the calibration measurements. After one year, some tests were repeated to check the drift and stability of calibration.This paper presents methodologies for calibration using data with significant autocorrelations. Autocorrelation in sensor measurements might be present when performing a frequent sampling. To obtain reliable results, sensor calibration methodologies must consider autocorrelation or serial correlation between subsequent measurements. Experimental design can be used to reduce the risk of highly autocorrelated measurement.Ordinary Least Squares Estimations should not be used when measurements are autocorrelated, as their central assumption is that the residuals are independent and identically distributed. Two alternative methods considering autocorrelation using a first-order Markov scaling are proposed: Maximum Likelihood and Restricted Maximum Likelihood Estimation (REML). REML has better compensations for the estimated parameters and the scaling parameters. Akaike information criterion was used to select the most significant parameters resulting in formaldehyde and temperature.The results were presented for only one of the eight sensors. According to EPA's recommendations, the tested formaldehyde LCSs were Tier III, supplementary monitoring. The LCS over-and under-estimated the values obtained by the reference sensor, but they presented very similar dynamic responses, indicating that LCS could be used to detect concentration changes after calibration.

Accelerating LMS-Based Equalization With Correlated Training Sequence in Bandlimited IM/DD Systems

As higher symbol rates are utilized in the intensity modulation and direct detection (IM/DD) scheme to meet the unrelenting growth of data traffic, overcoming the inter-symbol interference (ISI) induced by the limited bandwidth has become increasingly crucial. Channel equalization based on digital signal processing (DSP) is an effective solution, where least-mean squares (LMS) algorithm is adopted to adjust tap coefficients. However, the LMS algorithm usually has slow rate of convergence and requires lots of training symbols. This work proposes a novel training sequence to accelerate the LMS-based equalization. A first-order Markov chain (MC) is employed for sequence generation, which introduces correlation between samples and shapes signal spectrum. Compared with the conventional training sequence that consists of independent and identically distributed (i.i.d.) samples and has a white spectrum, the MC sequence enables faster convergence of tap coefficients and mean-squared error (MSE). Moreover, an experimental demonstration of a 43 Gbaud PAM-4 signal shows that the proposed sequence can achieve a lower pre-forward-error-correction (pre-FEC) bit error rate (BER) than that of the i.i.d. sequence with the same length. When the PAM-4 signal is transmitted over a 5-km standard single mode fiber (SSMF) with 6-dB system bandwidth of 10 GHz, more than 70% training sequence length reduction can be attained. When the fiber length is increased to 10 km and the signal suffers from severe power fading, more than 48% reduction can be achieved.

Stochastic residential occupancy schedules based on the American Time-Use Survey

The energy performance of residential buildings is closely correlated with occupants’ behavior and their schedules. Moreover, the energy consumption of heating, ventilation, and air conditioning (HVAC) systems, which are by far the biggest contributor to energy consumption in residential buildings, is controlled by the presence or absence of occupants in a building. Thus, accurate occupancy presence and activity profiles are important to determine actual energy demands and corresponding control schedules for residential buildings. Conventionally, building energy simulation tools typically use a single generic and static occupancy profile to represent a building’s occupancy schedule, regardless of day type or household size. However, literature in the field suggests that there is significant potential for improvement to allow for more flexibility and accuracy in the calculation of occupancy. The objective of this study is thus to develop a stochastic building occupancy model and propose it as a realistic replacement for the conventional generic static schedules. This three-state stochastic occupancy model is based on the 2019 American Time-Use Survey (ATUS) and considers the differences among weekdays, Saturdays, and Sundays. In this model, survey respondents are clustered based on the number of residents in their household and a first-order inhomogeneous Markov chain technique is used to generate occupancy presence schedules. The models’ results are then validated against the original ATUS data in terms of state probabilities, state durations, and number of state changes throughout the course of a day.

Quantification and Simulation of Landscape Anthropization around the Mining Agglomerations of Southeastern Katanga (DR Congo) between 1979 and 2090

In Southeastern Katanga, mining activities are (in)directly responsible for deforestation, ecosystem degradation and unplanned building densification. However, little is known about these dynamics at the local level. First, we quantify the landscape anthropization around four agglomerations of Southeastern Katanga (Lubumbashi, Likasi, Fungurume and Kolwezi) in order to assess the applicability of the Nature–Agriculture-Urbanization model based on the fact that natural landscapes are replaced by anthropogenic landscapes, first dominated by agricultural production, and then built-up areas. Secondly, we predict evolutionary trends of landscape anthropization by 2090 through the first-order Markov chain. Mapping coupled with landscape ecology analysis tools revealed that the natural cover that dominated the landscape in 1979 lost more than 60% of its area in 41 years (1979–2020) around these agglomerations in favor of agricultural and energy production, the new landscape matrix in 2020, but also built-up areas. These disturbances, amplified between 2010 and 2020, are more significant around Lubumbashi and Kolwezi agglomerations. Built-up areas which spread progressively will become the dominant process by 2060 in Lubumbashi and by 2075 in Kolwezi. Our results confirm the applicability of the Nature–Agriculture-Urbanization model to the tropical context and underline the urgency to put in place a territorial development plan and alternatives regarding the use of charcoal as a main energy source in order to decrease the pressure on natural ecosystems, particularly in peri-urban areas.

Evaluating changes in credit rating quality of U.S. farmer cooperatives

The default of a cooperative has significant implications on cooperative members and the agricultural supply chain. Therefore, monitoring and even predicting future changes in creditworthiness is of value to cooperative managers and their lenders. However, we know little about farmer cooperatives' credit profiles and behavior because their financial statements are seldom shared. Using a Moody’s credit rating model and a unique data set, this article estimates Markov chains to evaluate changes in farmer cooperatives’ credit quality. The unconditional (one-size-fits-all) probability matrix, as is typically estimated, is shown to not be appropriate in describing credit rating transitions. Results also show cooperatives do not exhibit rating change momentum since a downgrade is not likely to be followed by another downgrade in the next period. Credit ratings of farmer cooperatives with less than $20 million in net sales follow a first-order Markov chain with stationary probabilities and the cooperatives with net sales of more than $250 million follow a zero-order Markov chain. This article adds to the limited research available on the credit rating behavior of farmer cooperatives. Cooperative managers, directors, and lenders can utilize these findings to make more informed decisions to impact future credit ratings.

When History and Heterogeneity Matter: A Tutorial on the Impact of Markov Model Specifications in the Context of Colorectal Cancer Screening.

Markov models are used in health research to simulate health care utilization and disease states over time. Health phenomena, however, are complex, and the memoryless assumption of Markov models may not appropriately represent reality. This tutorial provides guidance on the use of Markov models of different orders and stratification levels in health decision-analytic modeling. Colorectal cancer (CRC) screening is used as a case example to examine the impact of using different Markov modeling approaches on CRC outcomes. This study used insurance claims data from commercially insured individuals in Oregon to estimate transition probabilities between CRC screening states (no screen, colonoscopy, fecal immunochemical test or fecal occult blood test). First-order, first-order stratified by sex and geography, and third-order Markov models were compared. Screening trajectories produced from the different Markov models were incorporated into a microsimulation model that simulated the natural history of CRC disease progression. Simulation outcomes (e.g., future screening choices, CRC incidence, deaths due to CRC) were compared across models. Simulated CRC screening trajectories and resulting CRC outcomes varied depending on the Markov modeling approach used. For example, when using the first-order, first-order stratified, and third-order Markov models, 30%, 31%, and 44% of individuals used colonoscopy as their only screening modality, respectively. Screening trajectories based on the third-order Markov model predicted that a higher percentage of individuals were up-to-date with CRC screening as compared with the other Markov models. The study was limited to insurance claims data spanning 5 y. It was not possible to validate which Markov model better predicts long-term screening behavior and outcomes. Findings demonstrate the impact that different order and stratification assumptions can have in decision-analytic models. This tutorial uses colorectal cancer screening as a case example to provide guidance on the use of Markov models of different orders and stratification levels in health decision-analytic models.Colorectal cancer screening trajectories and projected health outcomes were sensitive to the use of alternate Markov model specifications.Although data limitations precluded the assessment of model accuracy beyond a 5-y period, within the 5-y period, the third-order Markov model was slightly more accurate in predicting the fifth colorectal cancer screening action than the first-order Markov model.Findings from this tutorial demonstrate the importance of examining the memoryless assumption of the first-order Markov model when simulating health care utilization over time.

Bivariate copula-based CUSUM charts for monitoring conditional nonlinear processes with first-order autocorrelation

This paper considers the bivariate copula-based first-order Markov models to describe nonlinear dependence in autocorrelated data. Then the cumulative sum (CUSUM) charts based on log-likelihood ratio are developed to monitor the mean shifts. The performance of the proposed CUSUM charts is studied in both linear and nonlinear time series under weak, moderate and strong dependence by numerical simulations. Comparisons with the AR(1) model-based CUSUM and EWMA charts are present under the same range of scenarios. Simulation results show that the Clayton copula-based CUSUM charts are more effective to almost all the mean shifts and further have an obvious advantage in strong dependence. An example from chemical production is given to illustrate the implementation of the proposed control scheme.

Eighteen-month Course and Outcome of Adolescent Restrictive Eating Disorders: Persistence, Crossover, and Recovery

ABSTRACT Objective In adults, low-weight restrictive eating disorders, including anorexia nervosa (AN), are marked by chronicity and diagnostic crossover from restricting to binge-eating/purging. Less is known about the naturalistic course of these eating disorders in adolescents, particularly atypical AN (atyp-AN) and avoidant/restrictive food intake disorder (ARFID). To inform nosology of low-weight restrictive eating disorders in adolescents, we examined outcomes including persistence, crossover, and recovery in an 18-month observational study. Method We assessed 82 women (ages 10–23 years) with low-weight eating disorders including AN (n = 40; 29 restricting, 11 binge-eating/purging), atyp-AN (n = 26; 19 restricting, seven binge-eating/purging), and ARFID (n = 16) at baseline, nine months (9 M; 75% retention), and 18 months (18 M; 73% retention) via semi-structured interviews. First-order Markov modeling was used to determine diagnostic persistence, crossover, and recovery occurring at 9 M or 18 M. Results Among all diagnoses, the likelihood of remaining stable within a given diagnosis was greater than that of transitioning, with the greatest probability among ARFID (0.84) and AN-R (0.62). Persistence of BP and atypical presentations at follow-up periods was less stable (AN-BP probability 0.40; atyp-AN-R probability 0.48; atyp-AN-BP probability, 0.50). Crossover from binge-eating/purging to restricting occurred 72% of the time; crossover from restricting to binge-eating/purging occurred 23% of the time. The likelihood of stable recovery (e.g., recovery at both 9 M and 18 M) was between 0.00 and 0.36. Conclusion Across groups, intake diagnosis persisted in about two-thirds, and recovery was infrequent, underscoring the urgent need for innovative treatment approaches to these illnesses. Frequent crossover between AN and atyp-AN supports continuity between typical and atypical presentations, whereas no crossover to ARFID supports its distinction.

A Comparative Study of Geometric and Exponential Laws in Modelling the Distribution of Daily Precipitation Durations

Wet and dry duration modelling is critical in engineering hydrology. The goal of this study is to model and analyze the wet and dry duration distributions. For this intent, daily rainfall data for Kenitra station were used on the period from 1967 to 2017. To represent the distribution of wet and dry durations, First-order Markov chain, Second-order Markov chain, and truncated negative binomial distribution are applied to represent the distribution of wet and dry durations. To assess the data adherence to the proposed models, the Chi-square and Kolmogorov-Smirnov tests have been used. The Akaike information criterion is applied to determine the most effective model distribution. We go further to investigate the distribution of the number of wet and dry days over a k-day period. This law is implemented using an algorithm based on conditional laws. This work is completed by comparing the calculated moments of the three estimated models to the observed moments of the number of wet/dry days over k consecutive days. The study demonstrates the effectiveness of our method for modelling wet and dry daily precipitation durations.

Ex-ante effects of the 2018 Agricultural Improvement Act’s grassland initiative

The Grassland Conservation Initiative (GCI) was introduced under the United States 2018 Agricultural Improvement Act. The goal of the GCI is to conserve pasture and grassland. This study analyzes the ex-ante impacts of the GCI on crop returns and land-use acreage for Oklahoma farmers and ranchers. Oklahoma’s agricultural landscape is diverse. As with most agricultural policies focusing on land use, the distributional effects of GCI program will be conditional upon dominant land use types and grassland/pasture resources. Regions endowed with relatively more land resources in pasture and grassland are more likely to have more eligible acres, thereby potentially signaling to landowners in those regions opportunities to accrue land-holding rents. Land-use dynamics are estimated as a first-order Markov process. Not all regions benefit from this program. Findings suggest that aggregate net returns to cropland decrease. However, locations with above average pasture and grassland acreage experience gains in returns under certain policy assumptions. Counties experiencing gains from the GCI policy could experience higher returns to cattle and wheat compared to other counties with limited grassland and pasture acreage.

Analytic Method for Estimating Aircraft Fix Displacement From Gyroscope’s Allan-Deviation Parameters

The noise and drift requirements for a navigation-grade gyroscope are widely known, yet there is no simple analytic model of how the noise and drift of a gyroscope influence the fix displacement error (FDE) of an inertial navigation system (INS). This work derives simple analytical expressions for the cross-track and along-track errors of an aircraft whose INS consists solely of a three-axis gyroscope system with perfect knowledge of the vertical direction. The error signal of each gyroscope is Gaussian white noise and drift modeled as a first-order Markov random walk. These expressions provide a straightforward mean of calculating the FDE of an aircraft as a function of the flight duration, velocity, noise amplitude, drift amplitude, and drift’s time constant. These expressions are validated with Monte-Carlo simulations of long flights. This model quantifies the noise-versus-drift trade-off for a gyroscope in an inertial navigation system. It can save time when estimating the noise and drift that a gyroscope must exhibit to satisfy a given position-error requirement, or vice versa. They are used in particular to confirm the values, often cited without demonstration, of the noise and drift required to meet the maximum position error of an aircraft imposed by the Federal Aviation Administration’s required navigation performance 10 specification. Finally, it demonstrates that using the minimum in the measured Allan deviation of a gyroscope as a metric of the drift is incorrect, because it fails to capture the drift’s time constant. The proper metric is the maximum in the Allan deviation.

Gaussian estimation for non‐linear stochastic uncertain systems with time‐correlated additive noises and packet dropout compensations

This paper discusses the problem of state estimation for non-linear stochastic uncertain systems with time-correlated additive noise and packet dropout compensations. Both process noise and measurement noise are additive forms of noise that can be described as Gaussian first-order Markov processes. The stochastic uncertainty of the system is described by synchronous correlated white multiplicative noise. When the measurement information from the current epoch is lost, a compensation strategy based on measurement information prediction is used. First, the state information, process noise, and measurement noise are augmented to enable the construction of a new vector to be estimated, and the filter, predictor, and smoother of the system are designed using a Gaussian iterative estimation algorithm based on innovation analysis. The multiplicative noise in the process equation is estimated in real time. Second, a numerical implementation of the proposed algorithm is derived based on the third-degree spherical-radial Cubature Kalman filter(CKF). Finally, the proposed algorithm is compared with an extended Kalman filter (EKF) based on Taylor series expansion. The effectiveness of the proposed algorithm is verified by comparing the simulation results.

Real-time estimation of quality-related variable for dynamic and non-Gaussian process based on semisupervised Bayesian HMM

Real-time sensing of product quality-related key variables in industrial processes has long been a tough task due to technical or economical limitations. Data-driven soft sensing techniques prove to be promising solution to this problem. However, industrial data are complicated with compound complex characteristics, in particular the intractable process dynamics, non-Gaussian distributions and missing value of the quality variables, which render significant difficulties in the development of high-accuracy soft sensor. Given such vexed issues, this paper proposes a dynamic soft sensing method called ‘semisupervised Bayesian hidden Markov model (SsBHMM)’. In the SsBHMM, a semisupervised fully Bayesian regressive model structure is first designed, which accounts for the process dynamics using first-order Markov chain with hidden variables (HVs) and deals with the non-Gaussianities by mixture of Gaussians. Moreover, based on variational inference an efficient training algorithm is developed to learn parameters of the SsBHMM, which mines both labeled and unlabeled data such that the issue of missing value of quality variables can be dealt with. The performance of the SsBHMM is evaluated by a numerical example and an industrial low-temperature transformation unit, through which the advantages and the feasibility of the SsBHMM have been demonstrated.

Temporal-Structure-Aware Interference Cancellation for Asynchronous Cognitive IoT

In this paper, we investigate the interference cancellation problem for asynchronous cognitive Internet of Things (C-IoT) in the concurrent spectrum access (CSA) model. An iterative receiver together with a new clustering algorithm is employed to cancel the interference and recovers the desired signal. Since the C-IoT device and the primary user (PU) are non-cooperative, they are generally asynchronous at the symbol level and the interference signal received by the secondary user (SU) receiver (Rx) possesses a first-order Markov property over time. Therefore, we propose a new clustering algorithm named hidden Morkov model (HMM) based affinity propagation (AP) algorithm that combines the HMM and the AP algorithm to realize the clustering module. The proposed HMM-AP algorithm exploits the temporal-correlation information to improve the performance of the iterative receiver. We show that the receiver with the HMM-AP algorithm can work well in non-cooperative system and has a much better performance than the original AP.

Course of screening-based depression in young adults with a long type 1 diabetes duration: Prevalence and transition probabilities – A cohort study

AimsTo extend the current knowledge of the prevalence and course of screening-based depression (SBD) and its predictors in emerging adults with a long type 1 diabetes duration. MethodsA total of 487 young adults (64.7% women, mean age 24.0 years) who participated in a nationwide cohort study provided data on SBD (Patient Health Questionnaire (PHQ-9) score ≥ 10). We estimated the overall and age- and sex-specific prevalence of SBD, identified the associated covariates, and determined the transition probabilities between SBD states using adjusted first-order Markov transition models. ResultsThe prevalence of SBD was 17.7% in women and 7.0% in men. A total of 70.4% (95%-CI 57.4%; 80.8%) of the participants with SBD at the first screening still had SBD at the three-year follow-up. Of the subjects without SBD at baseline, 6.9% (4.9%; 9.8%) had SBD at follow-up. The main predictor of current SBD was previous SBD (OR 39.0 (15.4; 98.6)), followed by living in one’s own or in a shared apartment vs. living with both parents (OR: 2.75 (1.03; 7.36)). ConclusionsUsing an innovative analytical approach, emerging adults with a long diabetes duration demonstrated a moderate rate of incident SBD but a high rate of persistent SBD.