Hidden semi-Markov Model Research Articles

Detection of sedentary time and bouts using consumer-grade wrist-worn devices: a hidden semi-Markov model

BackgroundWrist-worn data from commercially available devices has potential to characterize sedentary time for research and for clinical and public health applications. We propose a model that utilizes heart rate in addition to step count data to estimate the proportion of time spent being sedentary and the usual length of sedentary bouts.MethodsWe developed and trained two Hidden semi-Markov models, STEPHEN (STEP and Heart ENcoder) and STEPCODE (STEP enCODEr; a steps-only based model) using consumer-grade Fitbit device data from participants under free living conditions, and validated model performance using two external datasets. We used the median absolute percentage error (MDAPE) to measure the accuracy of the proposed models against research-grade activPAL device data as the referent. Bland-Altman plots summarized the individual-level agreement with activPAL.ResultsIn OPTIMISE cohort, STEPHEN’s estimates of the proportion of time spent sedentary had significantly (p < 0.001) better accuracy (MDAPE [IQR] = 0.15 [0.06–0.25] vs. 0.23 [0.13–0.53)]) and agreement (Bias Mean [SD]=-0.03[0.11] vs. 0.14 [0.11]) than the proprietary software, estimated the usual sedentary bout duration more accurately (MDAPE[IQR] = 0.11[0.06–0.26] vs. 0.42[0.32–0.48]), and had better agreement (Bias Mean [SD] = 3.91[5.67] minutes vs. -11.93[5.07] minutes). With the ALLO-Active dataset, STEPHEN and STEPCODE did not improve the estimation of proportion of time spent sedentary, but STEPHEN estimated usual sedentary bout duration more accurately than the proprietary software (MDAPE[IQR] = 0.19[0.03–0.25] vs. 0.36[0.15–0.48]) and had smaller bias (Bias Mean[SD] = 0.70[8.89] minutes vs. -11.35[9.17] minutes).ConclusionsSTEPHEN can characterize the proportion of time spent being sedentary and usual sedentary bout length. The methodology is available as an open access R package available from https://github.com/limfuxing/stephen/. The package includes trained models, but users have the flexibility to train their own models.

Finite-Time Asynchronous H∞ Control for Non-Homogeneous Hidden Semi-Markov Jump Systems

This article explores the finite-time control problem associated with a specific category of non-homogeneous hidden semi-Markov jump systems. Firstly, a hidden semi-Markov model is designed to characterize the asynchronous interactions that occur between the true system mode and the controller mode, and emission probabilities are used to establish relationships between system models and controller modes. Secondly, a novel piecewise homogeneous method is introduced to tackle the non-homogeneous issue by taking into account the time-dependent transition rates for the jump rules between different modes of the system. Thirdly, an asynchronous controller is developed by applying Lyapunov theory along with criteria for stochastic finite-time boundedness, ensuring the specified H∞ performance level is maintained. Finally, the effectiveness of this method is verified through two simulation examples.

Hidden Semi-Markov Models for Semantic-Graph Language Modeling

Semantic communication is expected to play a critical role in reducing traffic load in future intelligent large-scale sensor networks. With advances in Machine Learning (ML) and Deep Learning (DL) techniques, design of semantically-aware systems has become feasible in recent years. This work focuses on improving the reliability of the semantic information represented in a graph-based language that has been recently proposed. Inaccuracies in the representation of the semantic information can arise due to multiple factors, such as algorithmic shortcomings or sensory errors, decreasing the performance of the semantic extractor. This study aims to model the temporal evolution of semantic information, represented using the graph language, to enhance its reliability. Each unique graph configuration is treated as a distinct state, leading to a Hidden Semi-Markov Model (HSMM) defined over the state space of the graph configurations. The HSMM formulation enables the integration of prior knowledge on the semantic signal into the graph sequences, enhancing the accuracy in identifying semantic innovations. Within the HSMM framework, algorithms designed for graph smoothing, semantic information fusion, and model learning are introduced. The efficacy of these algorithms in improving the reliability of the extracted semantic-graphs is demonstrated through simulations and video streams generated in the CARLA simulation environment.

Finite-time asynchronous control for semi-Markov jump systems with random uncertainties and actuator faults

This article is aimed at studying the finite-time reliable asynchronous control issue for continuous semi-Markov jump systems (S-MJSs) with actuator failures and randomly occurring uncertainties (ROUs). To characterize the asynchronous phenomenon between the system mode and the controller mode, the mode detector approach via a hidden semi-Markov model (HSM) is adopted. The transition probability (TP) matrix of the HSM is nonhomogeneous, which can be provided by a polytopic structure. By resorting the mode-dependent Lyapunov function, sufficient conditions are attained to guarantee the stochastic finite-time boundedness (SFTB) and H∞ performance of the closed-loop system. Then, according to a skillful matrix inequality transforming method, the reliable asynchronous control law is constructed. Finally, a tunnel diode circuit system and a mass-spring-damper mechanical system are proposed to demonstrate the superiority, effectiveness and practicality of the developed design method.

Revisiting alpha resting state dynamics underlying hallucinatory vulnerability: Insights from Hidden semi‐Markov Modeling

BackgroundResting state (RS) brain activity is inherently non-stationary. Hidden semi-Markov Models (HsMM) can characterize continuous RS data as a sequence of recurring and distinct brain states along with their spatio-temporal dynamics. New methodRecent explorations suggest that HsMM state dynamics in the alpha frequency band link to auditory hallucination proneness (HP) in non-clinical individuals. The present study aimed to replicate these findings to elucidate robust neural correlates of hallucinatory vulnerability. Specifically, we aimed to investigate the reproducibility of HsMM states across different data sets and within-data set variants as well as the replicability of the association between alpha brain state dynamics and HP. ResultsWe found that most brain states are reproducible in different data sets, confirming that the HsMM characterized robust and generalizable EEG RS dynamics on a sub-second timescale. Brain state topographies and temporal dynamics of different within-data set variants showed substantial similarities and were robust against reduced data length and number of electrodes. However, the association with HP was not directly reproducible across data sets. Comparison with existing methodsThe HsMM optimally leverages the high temporal resolution of EEG data and overcomes time-domain restrictions of other state allocation methods. ConclusionThe results indicate that the sensitivity of brain state dynamics to capture individual variability in HP may depend on the data recording characteristics and individual variability in RS cognition, such as mind wandering. Future studies should consider that the order in which eyes-open and eyes-closed RS data are acquired directly influences an individual’s attentional state and generation of spontaneous thoughts, and thereby might mediate the link to hallucinatory vulnerability.

HSMM multi-observations for prognostics and health management

An efficient maintenance policy allows for determining the current state of a system (diagnosis phase) and its future state (prognosis phase). We show in this paper that Markovian methods allow for obtaining many efficient indicators for the expert. To characterize the quality and robustness of these methods, we compared the Hidden Semi-Markov Model (HSMM) with the Hidden Markov Model (HMM). Several learning and decoding methods were included in the competition. A real case study was used as a particularly interesting working tool. The Remaining Useful Life (RUL) has also been included in this work.

Bagging in Hidden Semi-Markov Model for handwriting robot trajectory generation

Handwriting robots as an application of Imitation Learning (IL). However, most methods have poor accuracy of trajectory generation under task constraints, and models are less robust to changes in demonstration data. This paper proposes an IL algorithm named Bagging in Hidden Semi-Markov Model (BHSMM). The demonstration data is first divided into several sub-datasets, and each sub-dataset is encoded into several basic learning models by Hidden Semi-Markov Models (HSMM). Then the relationship between the task constraint points and the basic learning models is used to derive the weights. Finally, the trajectories adapted to the task constraints are generated based on the weights. We conducted experiments on the handwritten dataset LASA and compared the accuracy error with the original HSMM method. The results show that the BHSMM can generate trajectories that satisfy the position and velocity constraints and is more robust to changes in the demonstration data than the HSMM. In addition, satisfactory results are obtained in trajectory generation for real robot handwriting.

Improved Fault Detection Method based on HSMM

This paper proposes a fault detection method for multivariate statistical process control. The proposed method combines the Forward-Backward Hidden Semi-Markov Model (HSMM) and Principal Component Analysis (PCA). A stochastic automaton was used for multi-mode detection with many observation sequences. We used agglomerative clusters to find the initial parameters of HSMM. We allocated an adaptive threshold and a fixed threshold in each mode for fault detection with PCA, including Hotelling T2 statistic and squared predictive error (Q statistic). We simulated this method on the Tennessee Eastman Process (TEP). Some faults were designed with various runs and times of occurrence. The experimental results were compared with the Mixture Bayesian PCA, Hidden Markov Model (HMM), and HSMM methods. The results are robust with an efficient detection rate. This activity recommends ways to find action plans for multi-mode process monitoring in chemical plants.

Asynchronous Sliding-Mode Control for Discrete-Time Networked Hidden Stochastic Jump Systems With Cyber Attacks.

In this study, asynchronous sliding-mode control (SMC) for discrete-time networked hidden stochastic jump systems subjected to the semi-Markov kernel (SMK) and cyber attacks is investigated. Considering the statistical characteristic of the SMK, which is challenging to acquire in engineering, this study recognizes the SMK to be incomplete. Due to the mode mismatch between the original system and the control law in the operating process, a hidden semi-Markov model is proposed to describe the considered asynchronous situation. The main aim of this study is to construct an asynchronous SMC mechanism based on an incomplete SMK framework under the condition of random denial-of-service attacks so that the resulting closed-loop system can realize the mean-square stability. By virtue of the upper bound of the sojourn time in each mode, innovative techniques are developed for mean-square stability analysis under an incomplete SMK. Furthermore, an asynchronous SMC scheme is designed to achieve the reachability of the quasi-sliding mode. Finally, the effectiveness is verified using an electronic throttle model.

A machine learning approach for real-time cortical state estimation

Objective. Cortical function is under constant modulation by internally-driven, latent variables that regulate excitability, collectively known as ‘cortical state’. Despite a vast literature in this area, the estimation of cortical state remains relatively ad hoc, and not amenable to real-time implementation. Here, we implement robust, data-driven, and fast algorithms that address several technical challenges for online cortical state estimation. Approach. We use unsupervised Gaussian mixture models to identify discrete, emergent clusters in spontaneous local field potential signals in cortex. We then extend our approach to a temporally-informed hidden semi-Markov model (HSMM) with Gaussian observations to better model and infer cortical state transitions. Finally, we implement our HSMM cortical state inference algorithms in a real-time system, evaluating their performance in emulation experiments. Main results. Unsupervised clustering approaches reveal emergent state-like structure in spontaneous electrophysiological data that recapitulate arousal-related cortical states as indexed by behavioral indicators. HSMMs enable cortical state inferences in a real-time context by modeling the temporal dynamics of cortical state switching. Using HSMMs provides robustness to state estimates arising from noisy, sequential electrophysiological data. Significance. To our knowledge, this work represents the first implementation of a real-time software tool for continuously decoding cortical states with high temporal resolution (40 ms). The software tools that we provide can facilitate our understanding of how cortical states dynamically modulate cortical function on a moment-by-moment basis and provide a basis for state-aware brain machine interfaces across health and disease.

Formula omitted] control of singularly perturbed systems using deficient hidden semi-Markov model

This paper deals with the H∞ control of a class of stochastic multi-timescale systems, called Markov jump singularly perturbed systems. The hidden semi-Markov model is introduced to handle the situation when system modes are unavailable in semi-Markov systems. Such a model is assumed deficient, that is, it lacks knowledge about the emission probability, transition probability, and probability density function of the sojourn time. It is a more general case compared with works conducted with perfect transition information. Depending on whether a fast or slow sampling rate is used, the resulting discrete-time singularly perturbed system is modeled differently, for both of which the controller design is conducted. Furthermore, criteria expressed in terms of linear matrix inequalities (LMIs) are developed that guarantee the δ-error mean-square stability. An approach to estimate the upper bound on δ-error with incomplete information is provided, meanwhile, the relationship between system performance and the upper of singular perturbation parameter is also presented. Finally, two simulation examples using real-world systems are provided to corroborate the validity as well as the practical merits of the results.

Dynamic event-triggered fuzzy control for nonlinear singular semi-Markovian jump systems via state decomposition approach

This paper addresses the dynamic event-triggered fuzzy control problem of nonlinear singular semi-Markovian jump systems. Firstly, the interval type-2 fuzzy model is utilized to approximate nonlinear singular semi-Markovian jump systems. Secondly, by employing the state decomposition approach, the interval type-2 fuzzy singular system is decomposed into a differential equation and an algebraic equation. Thirdly, a new periodic dynamic event-triggered scheme with the adaptive variable is proposed to further relieve the communication pressure while ensuring the system performance. The hidden semi-Markov model is used to describe the phenomenon that the modes of fuzzy controller and the modes of original system are asynchronously operational. By designing the asynchronous event-triggered fuzzy controller based on quantized decomposed state vectors and constructing suitable Lyapunov-Krasovskii functional, sufficient conditions are derived to guarantee that the closed-loop fuzzy singular system is stochastically admissible. Finally, two simulation examples are provided to verify the validity and advantages of obtained results.

A Dynamic-Memory Event-Triggered Protocol to Multiarea Power Systems With Semi-Markov Jumping Parameter.

This work deals with the dynamic-memory event-triggered-based load frequency control issue for interconnected multiarea power systems (IMAPSs) associated with random abrupt variations and deception attacks. To facilitate the transient faults, a semi-Markov process is addressed to model the dynamic behavior of IMAPSs. In order to modulate transmission frequency, a novel area-dependent dynamic-memory event-triggered protocol (DMETP) is scheduled by resorting to a set of the historically released packets (HRPs), which ensures better dynamic performance. From the viewpoint of the defender, the randomly occurring deception attack is taken into account, which is regulated by a Bernoulli-distributed scalar. Benefitting from the DMETP scheduling, a novel framework of the memory-based asynchronous control strategy is formulated, in which the hidden semi-Markov model is adopted to reveal the mode mismatches. Based on the Lyapunov theory, sufficient conditions are established to ensure the stochastic stability of the resulting systems. In the end, the simulation result is presented to reveal the efficiency of the proposed dynamic-memory event-triggered-based approach.

A Hidden Semi-Markov Model for Predicting Production Cycle Time Using Bluetooth Low Energy Data

This study proposes a statistical model to characterize the temporal characteristics of an entire production process. The model utilizes received signal strength indicator (RSSI) data obtained from a Bluetooth low energy (BLE) network. A hidden semi-Markov model (HSMM) is formulated based on the characteristics of the production process, and the forward-backward algorithm is employed to re-estimate the probability distribution of state durations. The proposed method is validated through numerical, simulation, and real-world experiments, yielding promising results. The results show that the Kullback-Leibler divergence (KLD) score of 0.1843, while the simulation achieves an average vector distance score of 0.9740. The real-time experiment also shows a reasonable accuracy, with an average HSMM estimated throughput time of 30.48 epochs, compared to the average real throughput time of 33.99 epochs. Overall, the model serves as a valuable tool for predicting the cycle time and throughput time of a production line.

Unsupervised phoneme and word acquisition from continuous speech based on a hierarchical probabilistic generative model

Humans can divide the perceived continuous speech signals, which exhibit double articulation structure, into phonemes and words without explicit boundary points or labels and thus learn a language. In constructive developmental studies, learning the double articulation structure of speech signals is important for realizing robots with human-like language learning abilities. In this study, we propose a novel probabilistic generative model called the Gaussian process-hidden semi-Markov model-based double articulation analyzer (GP-HSMM-DAA), which can learn phonemes and words from continuous speech signals by hierarchically connecting two probabilistic generative models (PGMs), namely, the Gaussian process-hidden semi-Markov model and hidden semi-Markov model. In the proposed model, the parameters of each PGM are mutually and complementarily updated and learned, enabling accurate learning of the phonemes and words. The experimental results reveal that GP-HSMM-DAA can segment continuous speech into phonemes and words with higher accuracy than the conventional method.

Hidden Semi-Markov Models-Based Visual Perceptual State Recognition for Pilots.

Pilots' loss of situational awareness is one of the human factors affecting aviation safety. Numerous studies have shown that pilot perception errors are one of the main reasons for a lack of situational awareness without a proper system to detect these errors. The main objective of this study is to examine the changes in pilots' eye movements during various flight tasks from the perspective of visual awareness. The pilot's gaze rule scanning strategy is mined through cSPADE, while a hidden semi-Markov model-based model is used to detect the pilot's visuoperceptual state, linking the correlation between the hidden state and time. The performance of the proposed algorithm is then compared with that of the hidden Markov model (HMM), and the more flexible hidden semi-Markov model (HSMM) is shown to have an accuracy of 93.55%.

Residual useful life prognosis of equipment based on modified hidden semi-Markov model with a co-evolutional optimization method

Health diagnosis and prognosis of equipment is considered crucial for condition-based maintenance. This paper presents a prognosis health and monitoring scheme framework based on the modified hidden semi-Markov model (HSMM). First, a novel scalarization method for raw signals is developed based on the Weibull distribution with a short-time function window. Then, the conglutination coefficient and deterioration kernel are introduced into HSMM to simulate the inherent deterioration process of equipment. A co-evolutional algorithm based on genetic algorithm and salp swarm algorithm is proposed to estimate the parameters of the modified HSMM. A model base is established based on various sub-models of different faults for the health prognosis and diagnosis of equipment. The residual useful life can be calculated based on the historical whole useful life (WUL) data and the real-time status of equipment. Finally, the feasibility and effectiveness of the proposed model is verified on the turbofan engine dataset. The proposed scheme can provide a novel solution for the health prognosis of the non-vibration signal.

Physical activity in young children across developmental and health states: the ActiveCHILD study.

Evidence about physical activity of young children across developmental and health states is very limited. Using data from an inclusive UK cohort, ActiveCHILD, we investigated relationships between objectively measured physical activity, child development, social context, and health-related quality of life (HRQoL). Children (12-36 months), purposively sampled across health pathways, developmental abilities, and sociodemographic factors, were recruited through thirteen National Health Service organisations in England. Data were collected from 07/2017 to 08/2019 on: weekly physical activity (3-7 days) using waist-worn accelerometer (ActiGraph 3GTX); sociodemographics, parent actions, child HRQoL, and child development using questionnaires; and child health conditions using clinical records. A data-driven, unsupervised method, called hidden semi-Markov model (HSMM) segmented the accelerometery data and provided estimates of the total time spent active (any intensity) and very active (greater intensity) for each child. Relationships with the explanatory factors were investigated using multiple linear regression. Physical activity data were obtained for 282 children (56% females, mean age 21 months, 37.5% with a health condition) covering all index of multiple deprivation deciles. The patterns of physical activity consisted of two daily peaks, children spending 6.44 (SD=1.39) hours active (any intensity), of which 2.78 (SD=1.38) hours very active, 91% meeting WHO guidelines. The model for total time active (any intensity) explained 24% of variance, with mobility capacity the strongest predictor (β=0.41). The model for time spent very active explained 59% of variance, with mobility capacity again the strongest predictor (β=0.76). There was no evidence of physical activity explaining HRQoL. The findings provide new evidence that young children across developmental states regularly achieve mainstream recommended physical activity levels and challenges the belief that children with development problems need lower expectations for daily physical activity compared to peers. Advancing the rights of all children to participate in physical activity requires inclusive, equally ambitious, expectations for all. Niina Kolehmainen, HEE/NIHR Integrated Clinical Academic Senior Clinical Lecturer, NIHR ICA-SCL-2015-01-00, was funded by the NIHR for this research project. Christopher Thornton, Olivia Craw, Laura Kudlek, and Laura Cutler were also funded from this award. Tim Rapley is a member of the NIHR Applied Research Collaboration North East and North Cumbria, with part of his time funded through the related award (NIHR200173). The views expressed in this publication are those of the author(s) and not necessarily those of the NIHR, NHS, or the UK Department of Health and Social Care. The work of Kianoush Nazarpour is supported by Engineering and Physical Sciences Research Council (EPSRC), under grant number EP/R004242/2.

EDRN-based propagation model for popular microblog information detection

Microblogging platforms have become popular for sharing knowledge and communicating with others. Rumours and current trends can be identified promptly if popular information trends on these platforms can be detected in the early stages of dissemination. Microblog information propagation is primarily based on forwarders; the forwarded information is more likely to become popular if the forwarder's influence is greater and the time interval between two adjacent forwarders is shorter. Based on the explicit duration recurrent network (EDRN), a recurrent neural network, a new microblog information-propagation model for the early detection of popular information was presented. In this model, the observation is constructed based on the forwarder's influence level and the time interval between two adjacent forwarders. The influence level of the forwarder was estimated using the support vector machine classifier. To evaluate this model, experiments were conducted using Twitter and Weibo datasets. In the experiments, EDRN outperformed several popular recurrent neural networks, hidden semi-Markov models, and hidden Markov models, particularly when the training set was small. The timeliness evaluation results demonstrated that the model can effectively detect popular information at an early stage.

Using unsupervised machine learning to quantify physical activity from accelerometry in a diverse and rapidly changing population.

Accelerometers are widely used to measure physical activity behaviour, including in children. The traditional method for processing acceleration data uses cut points to define physical activity intensity, relying on calibration studies that relate the magnitude of acceleration to energy expenditure. However, these relationships do not generalise across diverse populations and hence they must be parametrised for each subpopulation (e.g., age groups) which is costly and makes studies across diverse populations and over time difficult. A data-driven approach that allows physical activity intensity states to emerge from the data, without relying on parameters derived from external populations, offers a new perspective on this problem and potentially improved results. We applied an unsupervised machine learning approach, namely a hidden semi-Markov model, to segment and cluster the raw accelerometer data recorded (using a waist-worn ActiGraph GT3X+) from 279 children (9-38 months old) with a diverse range of developmental abilities (measured using the Paediatric Evaluation of Disability Inventory-Computer Adaptive Testing measure). We benchmarked this analysis with the cut points approach, calculated using thresholds from the literature which had been validated using the same device and for a population which most closely matched ours. Time spent active as measured by this unsupervised approach correlated more strongly with PEDI-CAT measures of the child's mobility (R2: 0.51 vs 0.39), social-cognitive capacity (R2: 0.32 vs 0.20), responsibility (R2: 0.21 vs 0.13), daily activity (R2: 0.35 vs 0.24), and age (R2: 0.15 vs 0.1) than that measured using the cut points approach. Unsupervised machine learning offers the potential to provide a more sensitive, appropriate, and cost-effective approach to quantifying physical activity behaviour in diverse populations, compared to the current cut points approach. This, in turn, supports research that is more inclusive of diverse or rapidly changing populations.