First-order Markov Research Articles

Large-scale dependent multiple testing via higher-order hidden Markov models

ABSTRACT Taking into account the local dependence structure in large-scale multiple testing is expected to improve both the efficiency of the testing procedure and the interpretability of scientific findings. The hidden Markov model (HMM), as an effective model to describe the sequential dependence, has been successfully applied to large-scale multiple testing with local correlations. However, in many applications, the first-order Markov chain is not flexible enough to capture the complexity of local correlations. To address this issue, this paper proposes a novel multiple testing procedure that uses a higher-order Markov chain to better characterize local correlations among tests. The proposed procedure is validated by theoretical results and simulation studies, which show that it outperforms its competitors in terms of power. Finally, a real data analysis is presented to demonstrate the favorable performance of the proposed procedure.

Do discrepancies between subjective and objective health shift over time in later life? A markov transition model

Subjective health (SH) deteriorates less rapidly than objective health (OH) in older adults. However, scant evidence exists regarding if discrepancies between SH and OH shift in the same individuals over time. We explore whether such discrepancies change over time in a sample of older adults living in England, through a prospective, observational cohort study design. Using data from the English Longitudinal Study of Ageing, we followed a sample of 6803 older adults, aged 60+ years at baseline, over three waves of data collection (2002–2007), yielding two wave transitions. A ‘health asymmetry’ metric classified older adults into four categories at each wave, based on the level of agreement between their SH and OH scores (‘health pessimist’, ‘health optimist’, ‘good health realist’ and ‘poor health realist’). First-order Markov transition and generalised logit models yielded estimated transition probabilities and odds ratios for health asymmetry transitions over time. At baseline, 36.84% of the sample were ‘good health realists’, 33% were ‘poor health realists’, 14.54% were ‘health optimists’, and 15.62% were ‘health pessimists’. Good and poor health realists were likely to remain health realistic over time. Good health realists who did transition however, were likely to become health optimists. Subsequently, the proportion of health optimists in the sample increased over time. Health pessimists had a high probability of being lost to study attrition. In conclusion, health optimism (i.e. where SH is rated better than OH) becomes more prevalent over time, in later life. Future research should investigate if promoting positive SH appraisals among health pessimists and poor health realists can optimise health and survival outcomes.

Two-Stage Distributed Robust Optimization Scheduling Considering Demand Response and Direct Purchase of Electricity by Large Consumers

The integration of large-scale wind power into power systems has exacerbated the challenges associated with peak load regulation. Concurrently, the ongoing advancement of electricity marketization reforms highlights the need to assess the impact of direct electricity procurement by large consumers on enhancing the flexibility of power systems. In this context, this paper introduces a Distributed Robust Optimal Scheduling (DROS) model, which addresses the uncertainties of wind power generation and direct electricity purchases by large consumers. Firstly, to mitigate the effects of wind power uncertainty on the power system, a first-order Markov chain model with interval characteristics is introduced. This approach effectively captures the temporal and variability aspects of wind power prediction errors. Secondly, building upon the day-ahead scenarios generated by the Markov chain, the model then formulates a data-driven optimization framework that spans from day-ahead to intra-day scheduling. In the day-ahead phase, the model leverages the price elasticity of the demand matrix to guide consumer behavior, with the primary objective of maximizing the total revenue of the wind farm. A robust scheduling strategy is developed, yielding an hourly scheduling plan for the day-ahead phase. This plan dynamically adjusts tariffs in the intra-day phase based on deviations in wind power output, thereby encouraging flexible user responses to the inherent uncertainty in wind power generation. Ultimately, the efficacy of the proposed DROS method is validated through extensive numerical simulations, demonstrating its potential to enhance the robustness and flexibility of power systems in the presence of significant wind power integration and market-driven direct electricity purchases.

Statistical downscaling model for future projection of daily IDF relationship by Markov chain and kernel density estimator

ABSTRACT Changes in climate might have a significant impact on rainfall characteristics, including extreme rainfall. This study aims to project the future daily rainfall, preserving most of the rainfall characteristics, including extreme rainfall incorporating climate changes. This paper presents two hybrid semi-parametric statistical downscaling models for future projection of IDF curves. The precipitation flux from seven scenarios of ten GCMs and observed daily rainfall data are considered as predictors and predictand variables, respectively. At site, daily rainfall occurrence is modeled using a two-state first-order Markov chain. Rainfall amounts on each wet day are modelled using a univariate nonparametric kernel density estimator. Two types of amount generation models are presented in this study. The bounded model (KDE-SP) is developed, considering the support for the kernel distribution as positive. In the unbounded model (KDE-Ext), the wet days are reclassified as extreme and non-extreme rainy days. A significant increasing trend can be observed in the future projected intensity–duration–frequency relationships. The maximum increment using empirical distribution is observed as 93.21 and 80.93% on a 5-year return period in the far future for the SSP5-8.5 scenario, using KDE-Ext and KDE-SP models, respectively. Although both methods show similar results, the KDE-Ext model performs better in simulating extreme rainfall.

Information bounds for Gaussian copula parameter in stationary semiparametric Markov models

Let {Vt}t=1n be any univariate stationary first-order semiparametric Markov process generated from an unknown invariant marginal distribution and a bivariate Gaussian copula with unknown correlation coefficient α0∈(−1,1). We prove that 1−α02 is the semiparametric efficient variance bound for estimating the correlation parameter α0 in any Gaussian copula generated first-order stationary Markov models. Surprisingly, this variance bound is strictly larger than 1−α022 (when α0≠0), which is the semiparametric efficient variance bound derived by Klaassen and Wellner (1997) for estimating the correlation parameter using any i.i.d. data {(Xi,Yi)}i=1n generated from a bivariate Gaussian copula with two unknown marginal distributions.

The frequency distribution and stochastic analysis of the hydrological drought in northern Algeria

The objective of this study was to examine drought using the Streamflow Drought Index (SDI) at various time scales and its temporal evolution using monthly streamflow data from 1973 to 2009. Streamflow records were collected from a network of 14 hydrometric stations distributed throughout the study area. The Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC) were used to assess the quality of the adjustment. According to these criteria, the gamma law better suited the time scales of 3, 6, and 12 months, whereas the log-normal law was better suited to the scale of 9 months. The analysis of the Streamflow Drought Index in the three study basins (Middle and Upper Cheliff, Lower Cheliff, and the Mina) revealed that different classes of drought among 3, 6, 9, and 12-month time scales in the period of 1973 to 2009 had occurred, notably beginning in 1980. The frequency of 19 to 54% was found at all stations and in years marked by a mild drought. The moderate years had a frequency of 6 to 19%, while the severe and extreme years had a lower percentage (about 3 to 6%) in the study area. Two consecutive years of drought (D-D) were more likely in the Middle and Upper Cheliff basins (> 60%) for the 6, 9, and 12-month time scales, according to the transition of probability of first-order non-stationary Markov chain. On a three-month time scale, the transition probabilities (D-D) were greater than 50% in the Coastal basin and Lower Cheliff basin, as well as the Mina basin, and less than 50% in the Middle and Upper Cheliff basins.

A TDOA sequence estimation method of underwater sound source based on hidden Markov model

To address the time difference of arrival (TDOA) estimation problem in the passive positioning system with wideband underwater motion sound sources and distributed hydrophones, a hidden Markov model-based (HMM) TDOA sequence estimation method is proposed in this paper. The method estimates the TDOA with multi-frame output of cross-correlated signals received on hydrophones. The transfer equation of the TDOA is established as a first-order hidden Markov process by analyzing the motion characteristics of the moving sound source and delays obtained from different hydrophones. Dynamic assignment of the HMM parameters is proposed to address the inconsistent change rate of the TDOA. We then achieve an HMM expression of the TDOA sequence by fitting the transfer equation and dynamic assignment of parameters into the HMM. Then, the Viterbi algorithm (VA) is applied to distinguish the optimal sequence of the TDOA among ambiguous estimations. To deal with the problem of data loss or unreliable issues caused by interferences, a data prediction algorithm which could produce possible time delays is added to VA to avoid the impact of outliers on the estimation results. By utilizing multi-frame processing, the proposed method reduces the signal-to-noise ratio (SNR) requirement of single frames since it does not require accurate estimations of TDOA for each frame. Moreover, the method adapts to a lower SNR, which has significant advantages in terms of whole sequence estimation compared with common methods. The results from the simulations and lake experiments validated the proposed TDOA sequence estimation method.

Participation in Physical, Cognitive, and Social Activities and the Association with Mobility Disability Transitions in Older Adults.

Development of late-life mobility disability is a dynamic process of transitions between worsening and improving. We tested associations between participation in physical, social, and cognitive activity with mobility disability transitions. Participants (N=2,758, age 78.1 years [SD: 7.7]) from two Rush Alzheimer's Disease Center cohorts completed annual mobility disability questionnaires for 7.6 (SD: 4.4) years. First-order Markov transition models tested associations between baseline self-reported physical, social, and cognitive activity with bidirectional transitions in mobility disability score increases (worsening) and decreases (improving) between consecutive visits. Overall, 75.5% of participants experienced ≥1 transition among 18,318 pairs of consecutive visits-4,174 of which were worsening and 2,606 were improving transitions. Adjusting for covariates, higher participation in each activity type was associated with lower odds of worsening (physical OR=0.71, 95% CI: 0.67-0.75; social OR=0.64, 95% CI: 0.58-0.70; and cognitive OR=0.79, 95% CI: 0.74-0.85), and higher odds of improving (physical OR=1.20, 95% CI: 1.11-1.28; social OR=1.45, 95% CI: 1.30-1.61; and cognitive OR=1.12, 95% CI: 1.03-1.22) in separate models. In combined models, physical and social activity remained associated with worsening and improving; cognitive activity was only associated with worsening. Physical, social, and cognitive activity engagement contributes to lower odds of worsening mobility disability and may promote recovery.

Exponential stochastic volatility model with Laplace returns and its variants

This paper explores the exponential stochastic volatility model generated by first-order exponential Markov sequences to model financial time series. The stationary exponential Markov sequences used to generate the volatility model are the linear exponential autoregressive model, minification model, new exponential autoregressive time series model, and an exponential mixture of autoregressive and minification processes. The statistical properties of the models are studied and the estimation of model parameters is carried out using the generalized method of moments. An approximate Kalman filtering method and a non-linear filtering approach are employed to diagnose the models. To verify the correctness of the estimations, simulation studies are conducted, and two real datasets are used to demonstrate how the models are applied. Furthermore, a comparison of the suggested models is performed using the measures root mean square error, mean absolute error, and Value at Risk via a backtesting procedure.

Research on hydrological survey and forecasting model based on rolling sampling Markov chain

Abstract Hydrological survey is of great significance in flood disaster warning, hydropower engineering construction, and efficient utilization of watershed resources. Due to the seasonal and random fluctuations of hydraulic resources, it is difficult to accurately predict waterway runoff. Based on this, this article proposes a hydrological survey and prediction method based on rolling sampling Markov chain. Firstly, based on historical hydrological and meteorological data, establish a watershed runoff model and summarize and analyze the characteristics of the patterns; Then, based on the first-order Markov chain model, the relationship between adjacent days is considered. According to seasonal and meteorological factors, the historical data is rolling sampled in a 10 day sampling interval, and a multi state transition probability matrix is established to construct a year-round time-series runoff forecasting model; Finally, taking the Jinsha River Basin as an example, the annual runoff situation of the waterway was simulated. The calculation results have verified the effectiveness of the proposed method, indicating that it can simulate runoff well and provide certain guidance for watershed management and construction.

Inequality relations for NMR-based polymer homoblock analysis and extended application: Reanalysis of historical data on alginates, chitosans, homogalacturonans, and galactomannans

There has been a long-standing bottleneck in the quantitative analysis of the frequencies of homoblock polyads beyond triads using 1H and 13C NMR for linear polysaccharides, primarily because monosaccharides within a long homoblock share similar chemical environments due to identical neighboring units, resulting in indistinct NMR peaks. In this study, through rigorous mathematical induction, inequality relations were established that enabled the calculation of frequency ranges of homoblock polyads from historically reported NMR-derived frequency values of diads and/or triads of alginates, chitosans, homogalacturonans, and galactomannans. The calculated homoblock frequency ranges were then applied to evaluate three chain growth statistical models, including the Bernoulli chain, first-order Markov chain, and second-order Markov chain, for predicting homoblock frequencies in these polysaccharides. Furthermore, based on the mathematically derived inequality relations, a novel 2D array was constructed, enabling the graphical visualization of homoblock features in polysaccharides. It was demonstrated, as a proof of concept, that the novel 2D array, along with a 1D code generated from it, could serve as an effective feature engineering tool for polymer classification using machine learning algorithms.

A markov chain based photovoltaic power simulation method

Photovoltaic power generation has volatility and randomness, which affects the safety and reliability of the power system. In order to achieve accurate simulation of photovoltaic output, this paper proposes a photovoltaic power timing simulation method based on rolling sampled Markov chain model. Firstly, establish a photovoltaic output model and analyze the actual output characteristics; Then, based on the first-order Markov chain model, the relationship between adjacent days is considered, and a multistate transition probability matrix is established to construct an annual time series output model; Finally, based on the annual output data of a photovoltaic power station and historical meteorological monitoring data, an example is simulated to verify the effectiveness of the proposed method.

Forecasting the Power Generation Mix in Italy Based on Grey Markov Models

This study considers an application of the first-order Grey Markov Model to foresee the values of Italian power generation in relation to the available energy sources. The model is used to fit data from the Italian energy system from 2000 to 2022. The integration of Markovian error introduces a random element to the model, which is able now to capture inherent uncertainties and misalignments between the Grey Model predictions and the real data. This application provides valuable insights for strategic planning in the energy sector and future developments. The results show good accuracy of the predictions, which could provide powerful information for the effective implementation of energy policies concerning the evolution of energy demand in the country. Results show an improvement in the performance of more than 50% in terms of Root Mean Squared Error (RMSE) when the Markov chain is integrated in the analysis. Despite advancements, Italy’s 2032 energy mix will still significantly rely on fossil fuels, emphasizing the need for sustained efforts beyond 2032 to enhance sustainability.

The entropy rate of Linear Additive Markov Processes.

This work derives a theoretical value for the entropy of a Linear Additive Markov Process (LAMP), an expressive but simple model able to generate sequences with a given autocorrelation structure. Our research establishes that the theoretical entropy rate of a LAMP model is equivalent to the theoretical entropy rate of the underlying first-order Markov Chain. The LAMP model captures complex relationships and long-range dependencies in data with similar expressibility to a higher-order Markov process. While a higher-order Markov process has a polynomial parameter space, a LAMP model is characterised only by a probability distribution and the transition matrix of an underlying first-order Markov Chain. This surprising result can be explained by the information balance between the additional structure imposed by the next state distribution of the LAMP model, and the additional randomness of each new transition. Understanding the entropy of the LAMP model provides a tool to model complex dependencies in data while retaining useful theoretical results. To emphasise the practical applications, we use the LAMP model to estimate the entropy rate of the LastFM, BrightKite, Wikispeedia and Reuters-21578 datasets. We compare estimates calculated using frequency probability estimates, a first-order Markov model and the LAMP model, also considering two approaches to ensure the transition matrix is irreducible. In most cases the LAMP entropy rates are lower than those of the alternatives, suggesting that LAMP model is better at accommodating structural dependencies in the processes, achieving a more accurate estimate of the true entropy.

HMSL: Source localization based on higher-order Markov propagation

The widespread use of the Internet and social media has brought us great convenience, but it has also exposed us to a lot of false information and malicious attacks. It is vital to accurately locate the source of the harmful spread to prevent it from spreading further. Most previous studies have assumed that the propagation path is memoryless and always the shortest path. This assumption implies the first-order Markov property of propagation paths. This paper takes into account the higher-order Markov property of propagation paths in the source localization problem. Firstly, the problem of source localization based on observers is formulated. Then, we introduce the higher-order Markov property of propagation paths into the problem and propose a reaction–synchronization–diffusion model to model the propagation process on the higher-order network. On this basis, we build a framework named source localization based on higher-order Markov propagation (HMSL), which is compatible with traditional algorithms for source localization. After that, we conducted experiments on a real dataset and found that the HMSL has significant improvement in the source localization compared to the first-order network. Sensitivity analysis indicates that the degree of improvement is significantly influenced by the probability of infection and the proportion of higher-order nodes. Furthermore, we investigated the reason behind the improvement and found that the first-order network creates paths that do not exist within the raw data. When these fake paths are shorter than actual propagation paths, the length of propagation paths and estimated activation time of observers will be underestimated, thus decreasing the accuracy of source localization. The HMSL framework can solve this problem effectively.

Filtering and smoothing estimation algorithms from uncertain nonlinear observations with time-correlated additive noise and random deception attacks

ABSTRACT This paper discusses the problem of estimating a stochastic signal from nonlinear uncertain observations with time-correlated additive noise described by a first-order Markov process. Random deception attacks are assumed to be launched by an adversary, and both this phenomenon and the uncertainty in the observations are modelled by two sets of Bernoulli random variables. Under the assumption that the evolution model generating the signal to be estimated is unknown and only the mean and covariance functions of the processes involved in the observation equation are available, recursive algorithms based on linear approximations of the real observations are proposed for the least-squares filtering and fixed-point smoothing problems. Finally, the feasibility and effectiveness of the developed estimation algorithms are verified by a numerical simulation example, where the impact of uncertain observation and deception attack probabilities on estimation accuracy is evaluated.

Simulating sub-hourly rainfall data for current and future periods using two statistical disaggregation models: case studies from Germany and South Korea

Abstract. The simulation of fast-reacting hydrological systems often requires sub-hourly precipitation data to develop appropriate climate adaptation strategies and tools, i.e. upgrading drainage systems and reducing flood risks. However, these sub-hourly data are typically not provided by measurements and atmospheric models, and many statistical disaggregation tools are applicable only up to an hourly resolution. Here, two different models for the disaggregation of precipitation data from a daily to sub-hourly scale are presented. The first one is a conditional disaggregation model based on first-order Markov chains and copulas (WayDown) that keeps the input daily precipitation sums consistent within disaggregated time series. The second one is an unconditional rain generation model based on a double Poisson process (LetItRain) that does not reproduce the input daily values but rather generates time series with consistent rainfall statistics. Both approaches aim to reproduce observed precipitation statistics over different timescales. The developed models were validated using 10 min radar data representing 10 climate stations in Germany and South Korea; thus, they cover various climate zones and precipitation systems. Various statistics were compared, including the mean, variance, autocorrelation, transition probabilities, and proportion of wet period. Additionally, extremes were examined, including the frequencies of different thresholds, extreme quantiles, and annual maxima. To account for the model uncertainties, 1000-year-equivalent ensembles were generated by both models for each study site. While both models successfully reproduced the observed statistics, WayDown was better (than LetItRain) at reproducing the ensemble median, showing strength with respect to precisely refining the coarse input data. In contrast, LetItRain produced rainfall with a greater ensemble variability, thereby capturing a variety of scenarios that may happen in reality. Both methods reproduced extremes in a similar manner: overestimation until a certain threshold of rainfall and underestimation thereafter. Finally, the models were applied to climate projection data. The change factors for various statistics and extremes were computed and compared between historical (radar) information and the climate projections at a daily and 10 min scale. Both methods showed similar results for the respective stations and Representative Concentration Pathway (RCP) scenarios. Several consistent trends, jointly confirmed by disaggregated and daily data, were found for the mean, variance, autocorrelation, and proportion of wet periods. Further, they presented similar behaviour with respect to annual maxima for the majority of the stations for both RCP scenarios in comparison to the daily scale (i.e. a similar systematic underestimation).

Development of pedestrian collision avoidance strategy based on the fusion of Markov and social force models

Abstract. In urban traffic, accurate prediction of pedestrian trajectory and advanced collision avoidance strategy can effectively reduce the collision risk between intelligent vehicles and pedestrians. In order to improve the prediction accuracy of pedestrian trajectory and the safety of collision avoidance, a longitudinal and lateral intelligent collision avoidance strategy based on pedestrian trajectory prediction is proposed. Firstly, the process of a pedestrian crossing the road is considered as a combination of free motion described by first-order Markov model and the constrained motion presented by improved social force model. The predicted pedestrian trajectory is obtained by weighted fusion of the trajectories of the two models with a multiple linear regression algorithm. Secondly, according to the predicted pedestrian trajectory and time to collision (TTC) the longitudinal and lateral collision avoidance strategy is designed. The improved artificial potential field method is used to plan the lateral collision avoidance path in real time based on the predicted pedestrian position, and a fuzzy controller is constructed to obtain the desired deceleration of the vehicle. Finally, the pedestrian motion fusion model and the longitudinal and lateral collision avoidance strategy are verified by Prescan and Simulink co-simulation. The results show that the average displacement error (ADE) and final displacement error (FDE) of pedestrian trajectory based on pedestrian motion fusion model are smaller compared with a Markov model and improved social force model, and the proposed pedestrian collision avoidance strategy can effectively achieve longitudinal and lateral collision avoidance.

High-frequency stock market order transitions during the US-China trade war 2018: A discrete-time Markov chain analysis.

Statistical analysis of high-frequency stock market order transaction data is conducted to understand order transition dynamics. We employ a first-order time-homogeneous discrete-time Markov chain model to the sequence of orders of stocks belonging to six different sectors during the US-China trade war of 2018. The Markov property of the order sequence is validated by the Chi-square test. We estimate the transition probability matrix of the sequence using maximum likelihood estimation. From the heatmap of these matrices, we found the presence of active participation by different types of traders during high volatility days. On such days, these traders place limit orders primarily with the intention of deleting the majority of them to influence the market. These findings are supported by high stationary distribution and low mean recurrence values of add and delete orders. Further, we found similar spectral gap and entropy rate values, which indicates that similar trading strategies are employed on both high and low volatility days during the trade war. Among all the sectors considered in this study, we observe that there is a recurring pattern of full execution orders in the Finance & Banking sector. This shows that the banking stocks are resilient during the trade war. Hence, this study may be useful in understanding stock market order dynamics and devise trading strategies accordingly on high and low volatility days during extreme macroeconomic events.

Empirical evidence and computational assessment on design knowledge transferability

Abstract Developing an artificial design agent that mimics human design behaviors through the integration of heuristics is pivotal for various purposes, including advancing design automation, fostering human-AI collaboration, and enhancing design education. However, this endeavor necessitates abundant behavioral data from human designers, posing a challenge due to data scarcity for many design problems. One potential solution lies in transferring learned design knowledge from one problem domain to another. This article aims to gather empirical evidence and computationally evaluate the transferability of design knowledge represented at a high level of abstraction across different design problems. Initially, a design agent grounded in reinforcement learning (RL) is developed to emulate human design behaviors. A data-driven reward mechanism, informed by the Markov chain model, is introduced to reinforce prominent sequential design patterns. Subsequently, the design agent transfers the acquired knowledge from a source task to a target task using a problem-agnostic high-level representation. Through a case study involving two solar system designs, one dataset trains the design agent to mimic human behaviors, while another evaluates the transferability of these learned behaviors to a distinct problem. Results demonstrate that the RL-based agent outperforms a baseline model utilizing the first-order Markov chain model in both the source task without knowledge transfer and the target task with knowledge transfer. However, the model’s performance is comparatively lower in predicting the decisions of low-performing designers, suggesting caution in its application, as it may yield unsatisfactory results when mimicking such behaviors.