Learning Hidden Markov Models Research Articles

Code and Data Repository for "Learning Hidden Markov Models with Structured Transition Dynamics"

The goal of this code is to fit a Hidden Markov Model (HMM) to data with multiple linear constraints imposed on the HMM's transition and emission probability matrices via the LDKB-FISTA algorithm. Alongside our LDKB-FISTA, we have also uploaded the scripts for running Matlab-Fimincon method, as well as the traditional Baum Welch method. The code are written in Matlab.

Learning Hidden Markov Models with Structured Transition Dynamics

The hidden Markov model (HMM) provides a natural framework for modeling the dynamic evolution of latent diseases. The unknown probability matrices of HMMs can be learned through the well-known Baum–Welch algorithm, a special case of the expectation-maximization algorithm. In many disease models, the probability matrices possess nontrivial properties that may be represented through a set of linear constraints. In these cases, the traditional Baum–Welch algorithm is no longer applicable because the maximization step cannot be solved by an explicit formula. In this paper, we propose a novel approach to efficiently solve the maximization step problem under linear constraints by providing a Lagrangian dual reformulation that we solve by an accelerated gradient method. The performance of this approach critically depends on devising a fast method to compute the gradient in each iteration. For this purpose, we employ dual decomposition and derive Karush–Kuhn–Tucker conditions to reduce our problem into a set of single variable equations, solved using a simple bisection method. We apply this method to a case study on sports-related concussion and provide an extensive numerical study using simulation. We show that our approach is in orders of magnitude computationally faster and more accurate than other alternative approaches. Moreover, compared with other methods, our approach is far less sensitive with respect to increases in problem size. Overall, our contribution lies in the advancement of accurately and efficiently handling HMM parameter estimation under linear constraints, which comprises a wide range of applications in disease modeling and beyond. History: Accepted by Paul Brooks, Area Editor for Applications in Biology, Medicine, & Healthcare. Funding: This research was funded by [Grant R01DE028283] from the National Institute of Dental and Craniofacial Research, National Institutes of Health. G.-G. Garcia was also funded by the Georgia Clinical & Translational Science Alliance National Institutes of Health award [Grant UL1-TR002378]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0342 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0342 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Anomaly Detection and Classification in Multispectral Time Series Based on Hidden Markov Models

Monitoring agriculture from satellite remote sensing data, such as multispectral images, has become a powerful tool since it has demonstrated a great potential for providing timely and accurate knowledge of crops. Detecting anomalies in time series of multispectral remote sensing images for crop monitoring is generally performed using a large sample of historical data at a pixel level. Conversely, this article presents a framework for anomaly detection (AD), localization, and classification that exploits the temporal information contained in a given season at a parcel level to detect and localize outliers using hidden Markov models (HMMs). Specifically, the AD part is based on the learning of HMM parameters associated with unlabeled normal data that are used in a second step to detect abnormal crop parcels referred to as anomalies. The learned HMM can also be used in time segments to temporally localize the anomalies affecting the crop parcels. The detected and localized anomalies are finally classified using a supervised classifier, e.g., based on support vector machines. The proposed framework is applicable to images partially covered by clouds and can handle a set of crop parcels acquired in the same season bypassing problems due to crop rotations. Numerical experiments are conducted on synthetic and real data, where the real data correspond to vegetation indices extracted from several multitemporal Sentinel-2 images of rapeseed crops. The proposed approach is compared to standard AD methods yielding better detection rates with the advantage of allowing anomalies to be localized and characterized.

Using HMM and Reinforcement Learning for Building Smarter Trading Strategies

Samit Ahlawat demonstrates the superior performance of AI-based trading strategies over static rule-based strategies. Two AI algorithms are considered for constructing the trading strategy: hidden Markov model (HMM) and asynchronous advantage actor-critic (A3C) method

Learning hidden Markov models from aggregate observations

In this paper, we propose an algorithm for estimating the parameters of a time-homogeneous hidden Markov model (HMM) from aggregate observations. This problem arises when only the population level counts of the number of individuals at each time step are available, and one seeks to learn the individual HMM from these observations. Our algorithm is built upon the classical expectation–maximization algorithm and the recently proposed aggregate inference algorithm (Sinkhorn belief propagation). We present the parameter learning algorithm for two different settings of HMMs: one with discrete observations and one with continuous observations, and the algorithm exhibits convergence guarantees in both cases. Moreover, our learning framework naturally reduces to the standard Baum–Welch learning algorithm for HMMs when the population size is 1. The efficacy of our algorithm is demonstrated through several numerical experiments.

Phase transition for parameter learning of hidden Markov models.

We study a phase transition in parameter learning of hidden Markov models (HMMs). We do this by generating sequences of observed symbols from given discrete HMMs with uniformly distributed transition probabilities and a noise level encoded in the output probabilities. We apply the Baum-Welch (BW) algorithm, an expectation-maximization algorithm from the field of machine learning. By using the BW algorithm we then try to estimate the parameters of each investigated realization of an HMM. We study HMMs with n=4,8, and 16 states. By changing the amount of accessible learning data and the noise level, we observe a phase-transition-like change in the performance of the learning algorithm. For bigger HMMs and more learning data, the learning behavior improves tremendously below a certain threshold in the noise strength. For a noise level above the threshold, learning is not possible. Furthermore, we use an overlap parameter applied to the results of a maximum a posteriori (Viterbi) algorithm to investigate the accuracy of the hidden state estimation around the phase transition.

A graph-based big data optimization approach using hidden Markov model and constraint satisfaction problem

To address the challenges of big data analytics, several works have focused on big data optimization using metaheuristics. The constraint satisfaction problem (CSP) is a fundamental concept of metaheuristics that has shown great efficiency in several fields. Hidden Markov models (HMMs) are powerful machine learning algorithms that are applied especially frequently in time series analysis. However, one issue in forecasting time series using HMMs is how to reduce the search space (state and observation space). To address this issue, we propose a graph-based big data optimization approach using a CSP to enhance the results of learning and prediction tasks of HMMs. This approach takes full advantage of both HMMs, with the richness of their algorithms, and CSPs, with their many powerful and efficient solver algorithms. To verify the validity of the model, the proposed approach is evaluated on real-world data using the mean absolute percentage error (MAPE) and other metrics as measures of the prediction accuracy. The conducted experiments show that the proposed model outperforms the conventional model. It reduces the MAPE by 0.71% and offers a particularly good trade-off between computational costs and the quality of results for large datasets. It is also competitive with benchmark models in terms of the running time and prediction accuracy. Further comparisons substantiate these experimental findings.

Vision-Based Traffic Conflict Detection Using Trajectory Learning and Prediction

Although traffic conflict techniques have proven to be effective means for road safety analysis, they still suffer from incomplete conceptualization, observer subjectivity, and high data collection cost. To address these problems, video analysis has been increasingly applied to gain a better understanding of the behaviors of road users based on detailed motion data. However, the motion patterns underlying these data are rarely extracted to study the safety of their interactions. This article presents a vision-based method of traffic conflict detection through learning motion patterns from trajectories, for which an original algorithm was established through clustering and subsequent modeling. Using the extracted path and velocity information, we clustered trajectories hierarchically by applying an improved fuzzy $K$ -means algorithm with a modified Hausdorff distance. Each obtained cluster was taken as a labeled set to determine the structure and train the parameters of a hidden Markov model (HMM) that encoded the spatiotemporal characteristics of the trajectories as motion patterns. Based on the targeted trajectory predictions by the learned HMMs following the conflict development, a probabilistic model was developed to estimate the collision likelihood between vehicles to identify traffic conflicts. The experimental results obtained using actual traffic videos demonstrated the applicability of the algorithms for learning motion patterns and the feasibility of the approach for traffic conflict detection. The predicted trajectories were sufficiently accurate to calculate the collision probability, which was qualified as an indicator for measuring the conflict severity. These findings will have important implications for effective improvements in active road safety.

Probabilistic Load Forecasting Based on Adaptive Online Learning

Load forecasting is crucial for multiple energy management tasks such as scheduling generation capacity, planning supply and demand, and minimizing energy trade costs. Such relevance has increased even more in recent years due to the integration of renewable energies, electric cars, and microgrids. Conventional load forecasting techniques obtain single-value load forecasts by exploiting consumption patterns of past load demand. However, such techniques cannot assess intrinsic uncertainties in load demand, and cannot capture dynamic changes in consumption patterns. To address these problems, this paper presents a method for probabilistic load forecasting based on the adaptive online learning of hidden Markov models. We propose learning and forecasting techniques with theoretical guarantees, and experimentally assess their performance in multiple scenarios. In particular, we develop adaptive online learning techniques that update model parameters recursively, and sequential prediction techniques that obtain probabilistic forecasts using the most recent parameters. The performance of the method is evaluated using multiple datasets corresponding with regions that have different sizes and display assorted time-varying consumption patterns. The results show that the proposed method can significantly improve the performance of existing techniques for a wide range of scenarios.

Learning hidden Markov models with persistent states by penalizing jumps

Hidden Markov models are applied in many expert and intelligent systems to detect an underlying sequence of persistent states. When the model is misspecified or misestimated, however, it often leads to unrealistically rapid switching dynamics. To address this issue, we propose a novel estimation approach based on clustering temporal features while penalizing jumps. We compare the approach to spectral clustering and the standard approach of maximizing the likelihood function in an extensive simulation study and an application to financial data. The advantages of the proposed jump estimator include that it learns the hidden state sequence and model parameters simultaneously and faster while providing control over the transition rate, it is less sensitive to initialization, it performs better when the number of states increases, and it is robust to misspecified conditional distributions. The value of estimating the true persistence of the state process is illustrated through a simple trading strategy where improved estimates result in much lower transaction costs. Robustness is particularly critical when the model is part of a system used in production. Therefore, our proposed estimator significantly improves the potential for using hidden Markov models in practical applications.

Machine learning based cell association for mMTC 5G communication networks

With the advent of 5G communication networks, the number of devices on the core 5G network significantly increases.A5Gnetwork is a cloud native, massively connected internet of things (IoT) platform with a huge number of devices hosted on the network now known as massive machine type communication (mMTC). As ultra-low latency is pivotal in developing 5G communication, a proper cell association scheme is now required to meet the load and traffic needs of the new network, opposed to older cell association schemes which were based only on the reference signal received power (RSRP). This paper proposes an unsupervised machine learning algorithm, namely hidden Markov model (HMM) learning on the network's telemetry data, which is used to learn network parameters and select the best eNodeB for cell association. The proposed model uses an HMM learning followed by decoding for selecting the optimal cell for association.

Machine learning based cell association for mMTC 5G communication networks

With the advent of 5G communication networks, the number of devices on the core 5G network significantly increases.A5Gnetwork is a cloud native, massively connected internet of things (IoT) platform with a huge number of devices hosted on the network now known as massive machine type communication (mMTC). As ultra-low latency is pivotal in developing 5G communication, a proper cell association scheme is now required to meet the load and traffic needs of the new network, opposed to older cell association schemes which were based only on the reference signal received power (RSRP). This paper proposes an unsupervised machine learning algorithm, namely hidden Markov model (HMM) learning on the network's telemetry data, which is used to learn network parameters and select the best eNodeB for cell association. The proposed model uses an HMM learning followed by decoding for selecting the optimal cell for association.

은닉 마코프 모델을 이용한 심전도 QRS 검출 최적화에 관한 연구

The heart is the body"s circulatory organ that supplies blood to the body. An electrocardiogram is the best means of measuring and diagnosing abnormal conduction of the heart muscle. Therefore, to diagnose patients suspected of heart disease, a Holter monitoring system measuring electrocardiography for 24 hours is used by doctors to examine and diagnose patients. However, diagnosing innumerable Holter data entails considerable effort by doctors directly. If QRS can be detected by an automated diagnostic system, many Holter data could be classified without diagnosing it by clinicians directly. In this paper, we tried to detect the QRS of electrocardiogram using the Hidden Markov Model. For objective verification, the onsets and offsets of QRS, which classified by the specialist, in the QT-database, were used as the reference labels. The Mexican Hat mother function was used for the wavelet transform. To study how to optimize the learning of hidden Markov models, the experiment was conducted by changing the batch size of the training data sets and the scale of the wavelet mother function. During the verification process, the mean and standard deviation of the difference between QRS onset and offset obtained from the test data sets through the hidden Markov model and the reference label classified by a specialist were used. As a result, the batch size was found to have the best performance using all 84 training data sets, and the scale of the mother function was found to have the best performance using scale j = 2, 3, 4. When the mean and standard deviation of QRS complexes detected from the hidden Markov model was -8.2822ms, ±5.821476ms(p=0.99818) onsets, respectively and -2.9588ms, ±6.5662ms(p=0.99838) offsets, respectively with 84 batch sizes and all three scale mother functions were trained, the results of onset, offset standard deviation were improved average about 22.1%, 30.9% respectively when compared with other algorithms using QT-Database

Hybrid HMM/BLSTM system for multi-script keyword spotting in printed and handwritten documents with identification stage

In this paper, we propose a novel script-independent approach for word spotting in printed and handwritten multi-script documents. Since each writing type and script need to be processed using a specific spotting engine, the proposed system proceeds on two stages. The script identification is a preliminary stage that aims at recognizing on one level the writing type and the script of the input image document. Second, a specific word spotting method is used to spot query words in a large collection of documents. The proposed spotting system is based on deep bidirectional long short-term memory neural network and hidden Markov model (HMM) hybrid architecture. It takes advantage of DNN’s strong representation learning power and HMM’s sequential modeling ability. The global system has been evaluated on a mixed corpus of public databases such as KHATT, PKHATT for Arabic script and RIMES for Latin script. The experimental results on script identification and keyword spotting confirm the effectiveness of the proposed approach.

Semi-supervised learning of Hidden Markov Models for biological sequence analysis.

Hidden Markov Models (HMMs) are probabilistic models widely used in applications in computational sequence analysis. HMMs are basically unsupervised models. However, in the most important applications, they are trained in a supervised manner. Training examples accompanied by labels corresponding to different classes are given as input and the set of parameters that maximize the joint probability of sequences and labels is estimated. A main problem with this approach is that, in the majority of the cases, labels are hard to find and thus the amount of training data is limited. On the other hand, there are plenty of unclassified (unlabeled) sequences deposited in the public databases that could potentially contribute to the training procedure. This approach is called semi-supervised learning and could be very helpful in many applications. We propose here, a method for semi-supervised learning of HMMs that can incorporate labeled, unlabeled and partially labeled data in a straightforward manner. The algorithm is based on a variant of the Expectation-Maximization (EM) algorithm, where the missing labels of the unlabeled or partially labeled data are considered as the missing data. We apply the algorithm to several biological problems, namely, for the prediction of transmembrane protein topology for alpha-helical and beta-barrel membrane proteins and for the prediction of archaeal signal peptides. The results are very promising, since the algorithms presented here can significantly improve the prediction performance of even the top-scoring classifiers. Supplementary data are available at Bioinformatics online.

Tensor Train Spectral Method for Learning of Hidden Markov Models (HMM)

Abstract We propose a new algorithm for spectral learning of Hidden Markov Models (HMM). In contrast to the standard approach, we do not estimate the parameters of the HMM directly, but construct an estimate for the joint probability distribution. The idea is based on the representation of a joint probability distribution as an N-th-order tensor with low ranks represented in the tensor train (TT) format. Using TT-format, we get an approximation by minimizing the Frobenius distance between the empirical joint probability distribution and tensors with low TT-ranks with core tensors normalization constraints. We propose an algorithm for the solution of the optimization problem that is based on the alternating least squares (ALS) approach and develop its fast version for sparse tensors. The order of the tensor d is a parameter of our algorithm. We have compared the performance of our algorithm with the existing algorithm by Hsu, Kakade and Zhang proposed in 2009 and found that it is much more robust if the number of hidden states is overestimated.

Sequence Classification Using Third-Order Moments.

Model-based classification of sequence data using a set of hidden Markov models is a well-known technique. The involved score function, which is often based on the class-conditional likelihood, can, however, be computationally demanding, especially for long data sequences. Inspired by recent theoretical advances in spectral learning of hidden Markov models, we propose a score function based on third-order moments. In particular, we propose to use the Kullback-Leibler divergence between theoretical and empirical third-order moments for classification of sequence data with discrete observations. The proposed method provides lower computational complexity at classification time than the usual likelihood-based methods. In order to demonstrate the properties of the proposed method, we perform classification of both simulated data and empirical data from a human activity recognition study.

SUDMAD: Sequential and unsupervised decomposition of a multi‐author document based on a hidden markov model

Decomposing a document written by more than one author into sentences based on authorship is of great significance due to the increasing demand for plagiarism detection, forensic analysis, civil law (i.e., disputed copyright issues), and intelligence issues that involve disputed anonymous documents. Among existing studies for document decomposition, some were limited by specific languages, according to topics or restricted to a document of two authors, and their accuracies have big room for improvement. In this paper, we consider the contextual correlation hidden among sentences and propose an algorithm for Sequential and Unsupervised Decomposition of a Multi‐Author Document (SUDMAD) written in any language, disregarding topics, through the construction of a Hidden Markov Model (HMM) reflecting the authors' writing styles. To build and learn such a model, an unsupervised, statistical approach is first proposed to estimate the initial values of HMM parameters of a preliminary model, which does not require the availability of any information of author's or document's context other than how many authors contributed to writing the document. To further boost the performance of this approach, a boosted HMM learning procedure is proposed next, where the initial classification results are used to create labeled training data to learn a more accurate HMM. Moreover, the contextual relationship among sentences is further utilized to refine the classification results. Our proposed approach is empirically evaluated on three benchmark datasets that are widely used for authorship analysis of documents. Comparisons with recent state‐of‐the‐art approaches are also presented to demonstrate the significance of our new ideas and the superior performance of our approach.

Anomaly detection via short local trajectories

Trajectory provides an important motion cue in describing the behavior of the moving object and can be used effectively for anomaly detection. But traditional trajectories or tracklets used for analysis have limitations due to various tracking irregularities. In this paper, we propose a novel idea of detecting anomalies in a video, based on short history of a region in motion. These histories are defined as short local trajectories (SLT). In contrast to traditional tracklets, these SLTs are extracted for super-pixels belonging to foreground objects. This captures both spatial and temporal information of a candidate moving object. Furthermore, the proposed trajectory extraction is suitable across videos having different crowd density, occlusions, etc. The trajectories of persons/objects at a particular location under usual condition have certain attributes. Thus, we have used Hidden Markov Model (HMM) for characterizing the usual trajectory patterns for each defined region. The proposed algorithm takes SLTs as observations and measures the likelihood for each super-pixel of being anomaly based on learned HMMs. In order to avoid the influence of noisy trajectories, we have computed spatial consistency measure for each SLT based on the neighboring trajectories. Thus, anomalies detected by the proposed approach are highly localized as demonstrated from the experiments conducted on three anomaly datasets, namely UCSD Ped1, Ped2 and a newly proposed CHUK-Crowd Anomaly Dataset.

Weakly Supervised Learning of Hidden Markov Models for Spoken Language Acquisition

In this paper, a spoken command and control interface that acquires spoken language through demonstrations from the user is discussed. The user can train the system by uttering a command and subsequently demonstrating the required action through an alternative interface. From the demonstration, a bag of semantic concepts representation that represents which semantic concepts are present in the demonstration is extracted. In the previous work, we have proposed a method for learning words for these concepts by linking the bag of semantic concepts representation to a bag of features representation of the acoustics. In this method, the order in which the words occur is lost. However, in many cases, the order in which the words occur is important to be able to determine the correct action. In this paper, the vocabulary acquisition based on nonnegative matrix factorization is jointly trained with a hidden Markov model (HMM), making it possible to use the bag of concepts representation as a weak supervision for HMM learning. This model can better utilize the timing information to improve the results and the order in which the words occur is retained making it possible to learn vocabulary and grammar. The proposed system is tested on several command and control tasks and it is shown that for unimpaired speech the resulting system outperforms the system solely based on vocabulary acquisition.