Hierarchical Hidden Markov Model Research Articles

Killer whale respiration rates.

Measuring breathing rates is a means by which oxygen intake and metabolic rates can be estimated to determine food requirements and energy expenditure of killer whales (Orcinus orca) and other cetaceans. This relatively simple measure also allows the energetic consequences of environmental stressors to cetaceans to be understood but requires knowing respiration rates while they are engaged in different behaviours such as resting, travelling and foraging. We calculated respiration rates for different behavioural states of southern and northern resident killer whales using video from UAV drones and concurrent biologging data from animal-borne tags. Behavioural states of dive tracks were predicted using hierarchical hidden Markov models (HHMM) parameterized with time-depth data and with labeled tracks of drone-identified behavioural states (from drone footage that overlapped with the time-depth data). Dive tracks were sequences of dives and surface intervals lasting ≥ 10 minutes cumulative duration. We calculated respiration rates and estimated oxygen consumption rates for the predicted behavioural states of the tracks. We found that juvenile killer whales breathed at a higher rate when travelling (1.6 breaths min-1) compared to resting (1.2) and foraging (1.5)-and that adult males breathed at a higher rate when travelling (1.8) compared to both foraging (1.7) and resting (1.3). The juveniles in our study were estimated to consume 2.5-18.3 L O2 min-1 compared with 14.3-59.8 L O2 min-1 for adult males across all behaviours based on estimates of mass-specific tidal volume and oxygen extraction. Our findings confirm that killer whales take single breaths between dives and indicate that energy expenditure derived from respirations requires using sex, age, and behavioural-specific respiration rates. These findings can be applied to bioenergetics models on a behavioural-specific basis, and contribute towards obtaining better predictions of dive behaviours, energy expenditure and the food requirements of apex predators.

Hidden Markov models: Pitfalls and opportunities in ecology

Abstract Hidden Markov models (HMMs) and their extensions are attractive methods for analysing ecological data where noisy, multivariate measurements are made of a hidden, ecological process, and where this hidden process is represented by a sequence of discrete states. Yet, as these models become more complex and challenging to understand, it is important to consider what pitfalls these methods have and what opportunities there are for future research to address these pitfalls. In this paper, we review five lesser known pitfalls one can encounter when using HMMs or their extensions to solve ecological problems: (a) violation of the snapshot property in continuous‐time HMMs; (b) biased inference from hierarchical HMMs when applied to temporally misaligned processes; (c) sensitive inference from using random effects to partially pool across heterogeneous individuals; (d) computational burden when using HMMs to approximate models with continuous state spaces; and (e) difficulty linking the hidden process to space or environment. This review is for ecologists and ecological statisticians familiar with HMMs, but who may be less aware of the problems that arise in more specialised applications. We demonstrate how each pitfall arises, by simulation or example, and discuss why this pitfall is important to consider. Along with identifying the problems, we highlight potential research opportunities and offer ideas that may help alleviate these pitfalls. Each of the methods we review are solutions to current ecological research problems. We intend for this paper to heighten awareness of the pitfalls ecologists may encounter when applying these more advanced methods, but we also hope that by highlighting future research opportunities, we can inspire ecological statisticians to weaken these pitfalls and provide improved methods.

Phishing short URL detection based on link jumping on social networks

Nowadays, a large number of people frequently use social networks. Social networks have become important platforms for people to publish and obtain information. However, social networks have also become the main venue for hackers to initiate online fraud. Phishing is a common way used by hackers to launch online fraud on social networks. This paper proposes a method for detecting phishing short URL based on the link jumping. The method uses a hierarchical hidden Markov model with two-layer structure to describe the link jumping process after user clicking on short URL, so as to identify phishing short URL on social networks. The proposed method includes a training phase and an identification phase. In the identification phase, the average log-likelihood probability of the observation sequence is calculated. An experiment based on real datasets of Weibo is conducted to evaluate this method. The experiment results validate the effectiveness of this method.

Detecting bearish and bullish markets in financial time series using hierarchical hidden Markov models

Financial markets exhibit alternating periods of rising and falling prices. Stock traders seeking to make profitable investment decisions have to account for those trends, where the goal is to accurately predict switches from bullish to bearish markets and vice versa. Popular tools for modelling financial time series are hidden Markov models, where a latent state process is used to explicitly model switches among different market regimes. In their basic form, however, hidden Markov models are not capable of capturing both short- and long-term trends, which can lead to a misinterpretation of short-term price fluctuations as changes in the long-term trend. In this article, we demonstrate how hierarchical hidden Markov models can be used to draw a comprehensive picture of market behaviour, which can contribute to the development of more sophisticated trading strategies. The feasibility of the suggested approach is illustrated in two real-data applications, where we model data from the Deutscher Aktienindex and the Deutsche Bank stock. The proposed methodology is implemented in the R package fHMM, which is available on CRAN.

Toward Efficient Bayesian Approaches to Inference in Hierarchical Hidden Markov Models for Inferring Animal Behavior

The study of animal behavioral states inferred through hidden Markov models and similar state switching models has seen a significant increase in popularity in recent years. The ability to account for varying levels of behavioral scale has become possible through hierarchical hidden Markov models, but additional levels lead to higher complexity and increased correlation between model components. Maximum likelihood approaches to inference using the EM algorithm and direct optimization of likelihoods are more frequently used, with Bayesian approaches being less favored due to computational demands. Given these demands, it is vital that efficient estimation algorithms are developed when Bayesian methods are preferred. We study the use of various approaches to improve convergence times and mixing in Markov chain Monte Carlo methods applied to hierarchical hidden Markov models, including parallel tempering as an inference facilitation mechanism. The method shows promise for analysing complex stochastic models with high levels of correlation between components, but our results show that it requires careful tuning in order to maximize that potential.

Fraud Detection in Dynamic Interaction Network

Fraud detection from massive user behaviors is often regarded as trying to find a needle in a haystack. In this paper, we suggest abnormal behavioral patterns can be better revealed if both sequential and interaction behaviors of users can be modeled simultaneously, which however has rarely been addressed in prior work. Along this line, we propose a COllective Sequence and INteraction (COSIN) model, in which the behavioral sequences and interactions between source and target users in a dynamic interaction network are modeled uniformly in a probabilistic graphical model. More specifically, the sequential schema is modeled with a hierarchical Hidden Markov Model, and meanwhile it is shifted to the interaction schema to generate the interaction counts through Poisson factorization. A hybrid Gibbs-Variational algorithm is then proposed for efficient parameter estimation of the COSIN model. We conduct extensive experiments on both synthetic and real-world telecom datasets in different scales, and the results show that the proposed model outperforms some competitive baseline methods and is scalable. A case is further presented to show the precious explainability of the model.

Gerçek Ortamlarda Artımlı Öğrenme ile Gerçek Zamanlı İşitsel Sahne Analizi

Continual learning for scene analysis is a continuous process to incrementally learn distinct events, actions, and even noise models from past experiences using different sensory modalities. In this paper, an Auditory Scene Analysis (ASA) approach based on a continual learning system is developed to incrementally learn the acoustic events in a dynamically-changing domestic environment. The events being salient sound sources are localized by a Sound Source Localization (SSL) method to robustly process the signals of the localized sound source in the domestic scene where multiple sources can co-exist. For real-time ASA, audio patterns are segmented from the acoustic signal stream of the localized source for extraction of the audio features, and construction of a feature set for each pattern. The continual learning is employed via a time-series algorithm, Hidden Markov Model (HMM), on these feature sets from acoustic signals stemming from the sources. The learning process is investigated by conducting a variety of experiments to evaluate the performance of Unknown Event Detection (UED), Acoustic Event Recognition (AER), and continual learning using a Hierarchical HMM algorithm. The Hierarchical HMM consists of two layers: 1) a lower layer in which AER is performed using an HMM for each event and the event-wise likelihood thresholds; and 2) an upper layer in which UED is achieved by one HMM with a suspicion threshold through the audio features with their proto symbols stemming from the lower layer HMMs. We verified the effectiveness of the proposed system capable of continual learning, AER and UED in terms of False-Positive Rates, True-Positive Rates, recognition accuracy and computational time to meet the demands in a learning task of multiple events in real-time. The effectiveness of the AER system has been verified with high accuracy, and a short retraining time in real-time ASA having nine different sounds.

Context-Aware Smart Door Lock with Activity Recognition Using Hierarchical Hidden Markov Model

Context-Aware Security demands a security system such as a Smart Door Lock to be flexible in determining security levels. The context can be in various forms; a person’s activity in the house is one of them and is proposed in this research. Several learning methods, such as Naïve Bayes, have been used previously to provide context-aware security systems, using related attributes. However conventional learning methods cannot be implemented directly to a Context-Aware system if the attribute of the learning process is low level. In the proposed system, attributes are in forms of movement data obtained from a PIR Sensor Network. Movement data is considered low level because it is not related directly to the desired context, which is activity. To solve the problem, the research proposes a hierarchical learning method, namely Hierarchical Hidden Markov Model (HHMM). HHMM will first transform the movement data into activity data through the first hierarchy, hence obtaining high level attributes through Activity Recognition. The second hierarchy will determine the security level through the activity pattern. To prove the success rate of the proposed method a comparison is made between HHMM, Naïve Bayes, and HMM. Through experiments created in a limited area with real sensed activity, the results show that HHMM provides a higher F1-Measure than Naïve Bayes and HMM in determining the desired context in the proposed system. Besides that, the accuracies obtained respectively are 88% compared to 75% and 82%.

Detecting and Localizing Anomalies in Container Clusters Using Markov Models

Detecting the location of performance anomalies in complex distributed systems is critical to ensuring the effective operation of a system, in particular, if short-lived container deployments are considered, adding challenges to anomaly detection and localization. In this paper, we present a framework for monitoring, detecting and localizing performance anomalies for container-based clusters using the hierarchical hidden Markov model (HHMM). The model aims at detecting and localizing the root cause of anomalies at runtime in order to maximize the system availability and performance. The model detects response time variations in containers and their hosting cluster nodes based on their resource utilization and tracks the root causes of variations. To evaluate the proposed framework, experiments were conducted for container orchestration, with different performance metrics being used. The results show that HHMMs are able to accurately detect and localize performance anomalies in a timely fashion.

Technical Validation of an Automated Mobile Gait Analysis System for Hereditary Spastic Paraplegia Patients.

Hereditary spastic paraplegias (HSP) represents a group of orphan neurodegenerative diseases with gait disturbance as the predominant clinical feature. Due to its rarity, research within this field is still limited. Aside from clinical analysis using established scales, gait analysis has been employed to enhance the understanding of the mechanisms behind the disease. However, state of the art gait analysis systems are often large, immobile and expensive. To overcome these limitations, this paper presents the first clinically relevant mobile gait analysis system for HSP patients. We propose an unsupervised model based on local cyclicity estimation and hierarchical hidden Markov models (LCE-hHMM). The system provides stride time, swing time, stance time, swing duration and cadence. These parameters are validated against a GAITRite system and manual sensor data labelling using a total of 24 patients within 2 separate studies. The proposed system achieves a stride time error of -0.00 ±0.09s (correlation coefficient, r = 1.00) and a swing duration error of -0.67 ±3.27% (correlation coefficient, r = 0.93) with respect to the GAITRite system. We show that these parameters are also correlated to the clinical spastic paraplegia rating scale (SPRS) in a similar manner to other state of the art gait analysis systems, as well as to supervised and general versions of the proposed model. Finally, we show a proof of concept for this system to be used to analyse alterations in the gait of individual patients. Thus, with further clinical studies, due to its automated approach and mobility, this system could be used to determine treatment effects in future clinical trials.

Bowing Gestures Classification in Violin Performance: A Machine Learning Approach.

Gestures in music are of paramount importance partly because they are directly linked to musicians' sound and expressiveness. At the same time, current motion capture technologies are capable of detecting body motion/gestures details very accurately. We present a machine learning approach to automatic violin bow gesture classification based on Hierarchical Hidden Markov Models (HHMM) and motion data. We recorded motion and audio data corresponding to seven representative bow techniques (Détaché, Martelé, Spiccato, Ricochet, Sautillé, Staccato, and Bariolage) performed by a professional violin player. We used the commercial Myo device for recording inertial motion information from the right forearm and synchronized it with audio recordings. Data was uploaded into an online public repository. After extracting features from both the motion and audio data, we trained an HHMM to identify the different bowing techniques automatically. Our model can determine the studied bowing techniques with over 94% accuracy. The results make feasible the application of this work in a practical learning scenario, where violin students can benefit from the real-time feedback provided by the system.

Statistical structure of locomotion and its modulation by odors.

Most behaviors such as making tea are not stereotypical but have an obvious structure. However, analytical methods to objectively extract structure from non-stereotyped behaviors are immature. In this study, we analyze the locomotion of fruit flies and show that this non-stereotyped behavior is well-described by a Hierarchical Hidden Markov Model (HHMM). HHMM shows that a fly's locomotion can be decomposed into a few locomotor features, and odors modulate locomotion by altering the time a fly spends performing different locomotor features. Importantly, although all flies in our dataset use the same set of locomotor features, individual flies vary considerably in how often they employ a given locomotor feature, and how this usage is modulated by odor. This variation is so large that the behavior of individual flies is best understood as being grouped into at least three to five distinct clusters, rather than variations around an average fly.

A new driving condition identification method for heavy duty vehicles based on HHMM

Aiming to improve active safety of heavy duty vehicles, a new dynamic driving condition identification method is developed in this paper through incorporating Hierarchical Hidden Markov Models (HHMM) into the rollover warning system for heavy duty vehicles to assist the driver to be aware of the driving conditions. The corresponding data under typical driving conditions are first collected and then put into test with Student's t-test method and Grubbs's test method (T-G test method). The outliers filtered by T-G test from the data are detected and eliminated. K-Means algorithm, used to set up the rollover threshold value and Baum-Welch algorithm for optimising the proposed rollover warning model, are discussed in detail. Computer simulations under different driving conditions are carried out to verify the optimised HHMM. The simulation results demonstrate that the proposed driving condition identification method can effectively identify the driving status with a high accuracy under a variety of driving conditions and could be used for real-time rollover warning control.

A Hierarchical Hidden Markov Model Framework for Home Appliance Modeling

Correctly anticipating load characteristics of low voltage level is getting increased interest by distribution network operators. Energy disaggregation could be one of the potential approaches to exploit the massive amount of smart meter data to fulfill the task. Proper individual home appliance modeling is critical to the performance of NILM. In this paper, a hierarchical hidden Markov model (HHMM) framework to model home appliances is proposed. This model aims to provide better representation for those appliances that have multiple built-in modes with distinct power consumption profiles, such as washing machines and dishwashers. The dynamic Bayesian network representation of such an appliance model is built. A forward–backward algorithm, which is based on the framework of expectation maximization, is formalized for the HHMM fitting process. Tests on publically available data show that the HHMM and proposed algorithm can effectively handle the modeling of appliances with multiple functional modes, as well as better representing a general type of appliances. A disaggregation test also demonstrates that the fitted HHMM can be easily applied to a general inference solver to outperform conventional hidden Markov model in the estimation of energy disaggregation.

Intention Mining in Medical Process: A Case Study in Trauma Resuscitation.

In medical processes such as surgical procedures and trauma resuscitations, medical teams perform treatment activities according to underlying invisible goals or intentions. In this study, we present an approach to uncover these intentions from observed treatment activities. Developed on top of a hierarchical hidden Markov model (H-HMM), our approach can identify multi-level intentions. To accurately infer the H-HMM, we used state splitting method with maximum a posteriori probability (MAP) as the scoring function. We evaluated our approach in both qualitative and quantitative ways, using a case study of the trauma resuscitation process. This dataset includes 123 trauma resuscitation cases collected at a level 1 trauma center. Our results show our intention mining achieved an accuracy of 86.6% in classifying medical teams' intentions. This work is an exploration of unsupervised intention mining of complex real-world medical processes.

MONITORING FARMLAND LOSS CAUSED BY URBANIZATION IN BEIJING FROM MODIS TIME SERIES USING HIERARCHICAL HIDDEN MARKOV MODEL

Abstract. In this study, we proposed a method to map urban encroachment onto farmland using satellite image time series (SITS) based on the hierarchical hidden Markov model (HHMM). In this method, the farmland change process is decomposed into three hierarchical levels, i.e., the land cover level, the vegetation phenology level, and the SITS level. Then a three-level HHMM is constructed to model the multi-level semantic structure of farmland change process. Once the HHMM is established, a change from farmland to built-up could be detected by inferring the underlying state sequence that is most likely to generate the input time series. The performance of the method is evaluated on MODIS time series in Beijing. Results on both simulated and real datasets demonstrate that our method improves the change detection accuracy compared with the HMM-based method.

Segmentation of Gait Sequences in Sensor-Based Movement Analysis: A Comparison of Methods in Parkinson's Disease.

Robust gait segmentation is the basis for mobile gait analysis. A range of methods have been applied and evaluated for gait segmentation of healthy and pathological gait bouts. However, a unified evaluation of gait segmentation methods in Parkinson’s disease (PD) is missing. In this paper, we compare four prevalent gait segmentation methods in order to reveal their strengths and drawbacks in gait processing. We considered peak detection from event-based methods, two variations of dynamic time warping from template matching methods, and hierarchical hidden Markov models (hHMMs) from machine learning methods. To evaluate the methods, we included two supervised and instrumented gait tests that are widely used in the examination of Parkinsonian gait. In the first experiment, a sequence of strides from instructed straight walks was measured from 10 PD patients. In the second experiment, a more heterogeneous assessment paradigm was used from an additional 34 PD patients, including straight walks and turning strides as well as non-stride movements. The goal of the latter experiment was to evaluate the methods in challenging situations including turning strides and non-stride movements. Results showed no significant difference between the methods for the first scenario, in which all methods achieved an almost 100% accuracy in terms of F-score. Hence, we concluded that in the case of a predefined and homogeneous sequence of strides, all methods can be applied equally. However, in the second experiment the difference between methods became evident, with the hHMM obtaining a 96% F-score and significantly outperforming the other methods. The hHMM also proved promising in distinguishing between strides and non-stride movements, which is critical for clinical gait analysis. Our results indicate that both the instrumented test procedure and the required stride segmentation algorithm have to be selected adequately in order to support and complement classical clinical examination by sensor-based movement assessment.

Smart Annotation of Cyclic Data Using Hierarchical Hidden Markov Models.

Cyclic signals are an intrinsic part of daily life, such as human motion and heart activity. The detailed analysis of them is important for clinical applications such as pathological gait analysis and for sports applications such as performance analysis. Labeled training data for algorithms that analyze these cyclic data come at a high annotation cost due to only limited annotations available under laboratory conditions or requiring manual segmentation of the data under less restricted conditions. This paper presents a smart annotation method that reduces this cost of labeling for sensor-based data, which is applicable to data collected outside of strict laboratory conditions. The method uses semi-supervised learning of sections of cyclic data with a known cycle number. A hierarchical hidden Markov model (hHMM) is used, achieving a mean absolute error of 0.041 ± 0.020 s relative to a manually-annotated reference. The resulting model was also used to simultaneously segment and classify continuous, ‘in the wild’ data, demonstrating the applicability of using hHMM, trained on limited data sections, to label a complete dataset. This technique achieved comparable results to its fully-supervised equivalent. Our semi-supervised method has the significant advantage of reduced annotation cost. Furthermore, it reduces the opportunity for human error in the labeling process normally required for training of segmentation algorithms. It also lowers the annotation cost of training a model capable of continuous monitoring of cycle characteristics such as those employed to analyze the progress of movement disorders or analysis of running technique.

Decoding of individual finger movements from surface EMG signals using vector autoregressive hierarchical hidden Markov models (VARHHMM).

In this paper we present a novel method for predicting individual fingers movements from surface electromyography (EMG). The method is intended for real-time dexterous control of a multifunctional prosthetic hand device. The EMG data was recorded using 16 single-ended channels positioned on the forearm of healthy participants. Synchronously with the EMG recording, the subjects performed consecutive finger movements based on the visual cues. Our algorithm could be described in following steps: extracting mean average value (MAV) of the EMG to be used as the feature for classification, piece-wise linear modeling of EMG feature dynamics, implementation of hierarchical hidden Markov models (HHMM) to capture transitions between linear models, and implementation of Bayesian inference as the classifier. The performance of our classifier was evaluated against commonly used real-time classifiers. The results show that the current algorithm setup classifies EMG data similarly to the best among tested classifiers but with equal or less computational complexity.

Real-time performance-driven finger motion synthesis

This paper presents a method to estimate and synthesize the motion of a character’s fingers in real-time during the performance capture process. For the motion estimation and synthesis process, different motion datasets are used that contain the full-body motion of a character, including the motion of its fingers. The motion datasets have been pre-processed for being efficiently handled in real-time estimation process. During the runtime of the application, the system recognizes the actions of the user’s hands and assembles the necessary motion of the character’s fingers. By using a hierarchical Hidden Markov Model (HMM), the system learns the phase of the gestures as well as the progress of the motion. To eliminate the searching process of the most probable motion, prior constraints between segment states were assigned manually. During the runtime of our application, by using a forward HMM algorithm, the system synthesizes the necessary motion of a character’s fingers in real-time. The presented methodology is evaluated both numerically (system performance and estimation rate) and perceptually. The results show that, even when a reduced number of finger gestures are used, the synthesized motion can be described as perceptually consistent with the ground truth motion data.