Hidden Markov Model Classifier Research Articles

Improving the recognition of eating gestures using intergesture sequential dependencies.

This paper considers the problem of recognizing eating gestures by tracking wrist motion. Eating gestures are activities commonly undertaken during the consumption of a meal, such as sipping a drink of liquid or using utensils to cut food. Each of these gestures causes a pattern of wrist motion that can be tracked to automatically identify the activity. Previous works have studied this problem at the level of a single gesture. In this paper, we demonstrate that individual gestures have sequential dependence. To study this, three types of classifiers were built: 1) a K-nearest neighbor classifier which uses no sequential context, 2) a hidden Markov model (HMM) which captures the sequential context of subgesture motions, and 3) HMMs that model intergesture sequential dependencies. We built first-order to sixth-order HMMs to evaluate the usefulness of increasing amounts of sequential dependence to aid recognition. On a dataset of 25 meals, we found that the baseline accuracies for the KNN and the subgesture HMM classifiers were 75.8% and 84.3%, respectively. Using HMMs that model intergesture sequential dependencies, we were able to increase accuracy to up to 96.5%. These results demonstrate that sequential dependencies exist between eating gestures and that they can be exploited to improve recognition accuracy.

Automatic system to detect the type of voice pathology

Acoustic analysis is a noninvasive technique based on the digital processing of the speech signal. Acoustic analysis based techniques are an effective tool to support vocal and voice disease screening and especially in their early detection and diagnosis. Modern lifestyle has increased the risk of pathological voice problems. This work focuses on a robust, rapid and accurate system for automatic detection of normal and pathological speech and also to detect the type of pathology. This system employs non-invasive, inexpensive and fully automated measures of vocal tract characteristics and excitation information. Mel-frequency cepstral coefficients and linear prediction cepstral coefficients are used as acoustic features. The system uses Gaussian mixture model and hidden Markov model classifiers. Cerebral palsy, dysarthria, hearing impairments, laryngectomy, mental retardation, left side paralysis, quadriparesis, stammering, stroke, tumour in vocal tract are the types of pathologies considered in our experiments. From the experimental results, it is observed that to classify normal and pathological voice hidden Markov model with mel frequency cepstral coefficients with delta and acceleration coefficients is giving 94.44% efficiency. Likewise to identify the type of pathology Gaussian mixture model with mel frequency cepstral coefficients with delta and acceleration coefficients is giving 95.74% efficiency.

Automatic Check Digits Recognition for Arabic Using Multi-Scale Features, HMM and SVM Classifiers

We propose in this work two Automatic Arabic (Indian) digits recognition systems using a real-life dataset of 3000 bank checks. The systems extracts features from training-set images of 7390 isolated digits (0-9). These features are multi-scale in which they capture narrow, intermediate, and large-scale qualities of the image. The gradient features correspond to the narrow scale, the structural features correspond to the intermediate scale, and the concavity features correspond to the large-scale. These features are employed by two different statistical classifiers; Hidden Markov Models (HMM) and Support Vector Machines (SVM). The two independent recognition systems utilize the proficient CENPARMI Arabic bank check database for training and testing. In order to select the optimal parameters for feature extraction and for the HMM classifier, the CENPARMI training dataset is divided into training and verification subsets. After adapting the two systems’ parameters, they are tested on unobserved 3035 digit images. The average recognition rates for the HMM and SVM systems are 97.86% and 99.04%, respectively. The presented systems provides state-of-the-art recognition results on the CENPARMI database, as they reported a higher recognition rates when compared to twelve previously published systems, especially for the SVM system. After analyzing the classification errors, the authors conclude that some of these errors are inevitable as they are most probably attributed to errors in labeling the original database, distinct writing styles of certain digits, and genuine faults.

Hidden Markov Model Classification based on Empirical Frequencies of Observed Symbols

Given a sequence of observations, classification among two known hidden Markov models (HMMs) can be accomplished with a classifier that minimizes the probability of error (i.e., the probability of misclassification) by enforcing the maximum a posteriori probability (MAP) rule. For this MAP classifier, the a priori probability of error (before any observations are made) can be obtained, as a function of the length of the sequence of observations, by summing up the probability of error over all possible observation sequences of the given length, which is a computationally expensive task. In this paper, we obtain an upper bound on the probability of error of the MAP classifier. Our results are based on a suboptimal decision rule that ignores the order with which observations occur and relies solely on the empirical frequencies with which different symbols appear. We describe necessary and sufficient conditions under which this bound on the probability of error decreases exponentially with the length of the observation sequence. Apart from the usefulness of the suboptimal rule in bounding the probability of misclassification, its numerous advantages (such as low computational complexity, reduced storage requirements, and potential applicability to distributed or decentralized decision schemes) could prove a useful alternative to the MAP rule for HMM classification in many applications.

Fusion of HMM Classifiers, Based on x, y and (x, y) Signals, for the Recognition of Online Farsi Handwriting: a Large Lexicon Approach

Online handwriting is basically formed by a combination of horizontal and vertical trajectories. We employed decision fusion of semi-continuous hidden Markov model (HMM), classifiers, designed for x, y and (x, y) signals separately, to increase the recognition rate of online Farsi subwords. Because of large lexicon and few training samples for some classes, an embedded method was used to train HMM models of subwords. The result of segmentation of (x, y) signal was exploited to segment x and y signals to letters or sub-letters in the training phase. Different decision fusions of the three classifiers were investigated. The best result was achieved by applying the product rule to combine y and (x, y) classifiers.

FUZZY HIDDEN MARKOV CHAIN FOR WEB APPLICATIONS

Hidden Markov model (HMM) has become increasingly popular in the last several years. Real-world problems such as prediction of web navigation are uncertain in nature; in this case, HMM is less appropriate i.e., we cannot assign certain probability values while in fuzzy set theory everything has elasticity. In addition to that, a theory of possibility on fuzzy sets has been developed to handle uncertainity. Thus, we propose a fuzzy hidden Markov chain (FHMC) on possibility space and solve three basic problems of classical HMM in our proposed model to overcome the ambiguous situation. Client's browsing behavior is an interesting aspect in web access. Analysis of this issue can be of great benefit in discovering user's behavior in this way we have applied our proposed model to our institution's website ( www.ssn.edu.in ) to identify how well a given model matches a given observation sequence, next to find the corresponding state sequence which is the best to explain the given observation sequence and then to attempt to optimize the model parameters so as to describe best how a given observation sequence comes about. The solution of these problems help us to know the authenticity of the website.

Harmony search-based hidden Markov model optimization for online classification of single trial eegs during motor imagery tasks

This paper presents an improved method based on single trial EEG data for the online classification of motor imagery tasks for brain-computer interface (BCI) applications. The ultimate goal of this research is the development of a novel classification method that can be used to control an interactive robot agent platform via a BCI system. The proposed classification process is an adaptive learning method based on an optimization process of the hidden Markov model (HMM), which is, in turn, based on meta-heuristic algorithms. We utilize an optimized strategy for the HMM in the training phase of time-series EEG data during motor imagery-related mental tasks. However, this process raises important issues of model interpretation and complexity control. With these issues in mind, we explore the possibility of using a harmony search algorithm that is flexible and thus allows the elimination of tedious parameter assignment efforts to optimize the HMM parameter configuration. In this paper, we illustrate a sequential data analysis simulation, and we evaluate the optimized HMM. The performance results of the proposed BCI experiment show that the optimized HMM classifier is more capable of classifying EEG datasets than ordinary HMM during motor imagery tasks.

Classification of Seismic Volcanic Signals Using Hidden-Markov-Model-Based Generative Embeddings

The automated classification of seismic volcanic signals has been faced with several different pattern recognition approaches. Among them, hidden Markov models (HMMs) have been advocated as a cost-effective option having the advantages of a straightforward Bayesian interpretation and the capacity of dealing with seismic sequences of different lengths. In the volcano seismology scenario, HMM-based classification schemes were only based on a standard and purely generative scheme, i.e., the Bayes rule: training an HMM per class and classifying an incoming seismic signal according to the class whose model shows the highest likelihood. In this paper, a novel HMM-based classification approach for pretriggered seismic volcanic signals is proposed. The main idea is to enrich the classical HMM scheme with a discriminative step that is able to recover from situations when the classical Bayes classification rule is not sufficient. More in detail, a generative embedding scheme is used, which employs the models to map the signals into a vector space, which is called generative embedding space. In such a space, any discriminative vector-based classifier can be applied. A thorough set of experiments, which is carried out on pretriggered signals recorded at Galeras Volcano in Colombia, shows that the proposed approach typically outperforms standard HMM-based classification schemes, also in some cross-station cases.

Abstract 5133: Inference of tumor phylogenetic markers from large copy number datasets.

Abstract Understanding tumors as evolutionary systems is an important area of study with far-reaching implications in diagnostic and treatment paradigms. Computational phylogenetics is a valuable method for inferring tumor evolution in terms of evolutionary trees, phylogenies, where paths in a tree correspond to possible tumor progression pathways. The location of specific cell-types and patient samples in the tree provide information on tumor sub-types and development of heterogeneity. We previously developed a tumor phylogeny inference pipeline for array comparative genome hybridization (aCGH)-based tumor copy number profiles. Steps in the pipeline included extraction of robust progression markers from the data, which could differentiate stages of tumor evolution or the different paths in the tree, and assigning amplification states to the inferred markers in those stages. We introduced a novel multi-sample model for amplicon identification and calling, HMMCNA, which jointly extracted markers from and assigned amplification states to small sets of tumor aCGH profiles. HMMCNA employs a Hidden Markov Model (HMM), a probabilistic model, to classify data into normal and amplified states based on an underlying distribution for the two copy number states and a hidden state space of possible amplification states. We assumed two possible amplification states per sample: normal (0) or amplified (1). Joint segmentation and calling is performed by identifying a most likely sequence of amplification states across all genomic sites probes and samples. This approach limits in the number of samples the HMM can handle since the number of possible hidden amplification states increases exponentially with the number of samples. Here, we present an extension of the approach to handle large datasets. We incorporate a heuristic prior to the HMM classification to reduce the hidden state space by first screening out amplification states not strongly supported at any individual genome coordinates. The introduction of this heuristic reduces the state space on average by 99%. We further reduce the set of possible amplification states based on the frequency of occurrence of the states by only allowing those states occuring at multiple aCGH probes or array genome coordinate. This step accounts for the presence of random noise in the data and gives a further reduction of 80%. We demonstrate the method on a breast tumor aCGH dataset comprising copy number profiles derived from sectioned biopsy samples (NCBI GEO GSE16672, Navin et al., 2010). Our method was able to quickly segment the data into sets of robust normal and amplified segments suitable for downstream phylogeny building. The amplicons inferred carried several known markers of tumor progression. Further steps include tuning the parameters of the HMM to handle noise-levels across different datasets. Citation Format: Ayshwarya Subramanian, Stanley Shackney, Russell Schwartz. Inference of tumor phylogenetic markers from large copy number datasets. [abstract]. In: Proceedings of the 104th Annual Meeting of the American Association for Cancer Research; 2013 Apr 6-10; Washington, DC. Philadelphia (PA): AACR; Cancer Res 2013;73(8 Suppl):Abstract nr 5133. doi:10.1158/1538-7445.AM2013-5133

Towards More Accurate Clutering Method by Using Dynamic Time Warping

An intrinsic problem of classifiers based on machine learning (ML) methods is that their learning time grows as the size and complexity of the training dataset increases. For this reason, it is important to have efficient computational methods and algorithms that can be applied on large datasets, such that it is still possible to complete the machine learning tasks in reasonable time. In this context, we present in this paper a more accurate simple process to speed up ML methods. An unsupervised clustering algorithm is combined with Expectation, Maximization (EM) algorithm to develop an efficient Hidden Markov Model (HMM) training. The idea of the proposed process consists of two steps. In the first step, training instances with similar inputs are clustered and a weight factor which represents the frequency of these instances is assigned to each representative cluster. Dynamic Time Warping technique is used as a dissimilarity function to cluster similar examples. In the second step, all formulas in the classical HMM training algorithm (EM) associated with the number of training instances are modified to include the weight factor in appropriate terms. This process significantly accelerates HMM training while maintaining the same initial, transition and emission probabilities matrixes as those obtained with the classical HMM training algorithm. Accordingly, the classification accuracy is preserved. Depending on the size of the training set, speedups of up to 2200 times is possible when the size is about 100.000 instances. The proposed approach is not limited to training HMMs, but it can be employed for a large variety of MLs methods.

Sign language translation system based on micro-inertial measurement units and ZigBee network

Chinese sign language has been proved as an effective communication tool for deaf people. In this paper, we present a novel translation system, which can capture human gestures through micro-inertial measurement units (IMUs) and translate the gestures into specific meanings accordingly. Each micro-IMU consists of a 3D accelerometer and gyroscope. A micro-controller and ZigBee network were used to acquire data simultaneously and wirelessly. Ten types of basic Chinese sign language movements including what, how, work, today, happy, please, book, body, clothes and and were collected and stored to form a motion-sensing database. A discrete cosine transform (DCT) was performed to extract the effective features from the original data, while a hidden Markov model (HMM) was used to train the database in order to form an HMM classifier. Testing samples were used to test the HMM classifier. Different sign languages were recognized through the HMM classifier and subsequent translation processes were performed. Experimental results showed that the correct recognition rate ranges from 95% to 100% for the 10 sign language movements, and the overall correction rate is 98%. With more micro-electro-mechanical system (MEMS) sensing motes adding to the interpretation system, the performance will be enhanced.

Recognition of Offline Handwritten Chinese Characters of Amount in Words Based on Integrated Features and HMM

A recognition method of offline handwritten Chinese characters of amount in words is presented. The method uses elastic mesh strategy to divide character images written by special men into meshes, and extracts directional element and key point features in every mesh to produce a vector. Based on independent Hidden Markov Model classifiers, this paper uses voting rule to integrate the Hidden Markov Model classifiers. The experimental results show that this method has a relative high recognition rate.

Quality Monitoring of Surface Roughness and Roundness Using Hidden Markov Model

The surface roughness and roundness(SRR) are widely used indexes of mechanical product quality. How to implement the SRR monitoring is a crucial task. In this study, the hidden Markov models (HMMs) and the cutting vibration signals are applied to monitor the SRR in variant cutting conditions. Unlike most of the prior work only to reveal one element of the geometric specifications, based on the theoretical analysis of the influence of tool vibration displacement on the SRR, the vibration energy characteristic(VEC) is determined to serve as the characteristic for monitoring surface roughness(Ra) and roundness(Rd) synchronously.Which make up the insufficiency of the comprehensive monitoring of workpiece quality. Moreover, although classical hidden Markov models (HMMs) have been successfully used for fault diagnostics of mechanical systems, this method based on recognition rate is becoming unreliable to monitor the accuracy of the workpiece. Hence, the HMM-based judgment matrix method is proposed and it is tested and validated successfully using for SRR monitoring through a series of experiments.

Fault Diagnostics of Helicopter Gearboxes Based on Multi-Sensor Mixtured Hidden Markov Models

Accurate identification of faults in gearboxes is of vital importance for the safe operation of helicopters. Although hidden Markov models (HMMs) with Gaussian observations have been successfully used for fault diagnostics of mechanical systems, a Gaussian HMM must assume that the observation sequence is generated from a Gaussian process. Conversely, vibration signals from helicopter gearboxes are often non-Gaussian and non-stationary. Also, it always needs to use multi-sensors for more accurate fault diagnostics in practice. Thus, a classical Gaussian HMM may not meet the need of helicopter gearboxes, and it needs to study novel HMMs to model multi-sensor, non-Gaussian signals. This paper presents a multi-sensor mixtured HMM (MSMHMM), which is built on multi-sensor signals. For a MSMHMM, each sensor signal will be considered as the mixture of non-Gaussian sources, so it can depict non-Gaussian observation sequences very well. Then, learning mechanisms of MSMHMM parameters are formulated in detail based on the expectation-maximization (EM) algorithm and a framework of MSMHMM-based fault diagnostics is proposed. In the end, the proposed method is validated on a helicopter gearbox, and the results are very exciting.

Semi-supervised context adaptation: case study of audience excitement recognition

To recognise just the same human reaction (for example, a strong excitement) in different contexts, customary behaviours in these contexts have to be taken into account; e.g. a happy sport audience may be cheering for long time, while a happy theatrical audience may produce only short bursts of laughter in order to not interrupt the performance. Tailoring recognition algorithms to contexts can be achieved by building either a context-specific or a generic system. The former is individually trained for each context to recognise sets of characteristic responses, whereas the latter—in contrast to the context-specific one—adapts to the context via significantly more lightweight modification of parameters. This paper follows the latter way and proposes a simple modification of a hidden Markov model (HMM) classifier that enables end users to adapt the generic system to a context or a personal perception of an annotator by labelling a fairly small number of data samples of each context. For better adaptability to the limited number of the user’s annotations, the proposed semi-supervised HMM classifier employs the maximum posterior marginal, rather than the more conventional maximum a posteriori decision rule. The proposed user- and context-adaptable semi-supervised HMM classifier was tested on recognising excitement of a show audience in three contexts (a concert hall, a circus, and a sport event), differing in how the excitement is expressed. In our experiments the proposed classifier recognised reactions of a non-neutral audience with 10% higher accuracy than the conventional HMM and support vector machine based classifiers.

Fracture Images Classification Based on Fractional Cosine Transform and Markov Mode

Fracture images automatic classification and recognition is an important one of fracture failure intelligent diagnosis, and in which feature extraction is a key issue. In this paper, fractional cosine transform, which is a useful time-frequency analysis method, is used in feature extraction of fracture images, and then the classification of fatigue, dimples, intergranular and cleavage is performed by Hidden markov model (HMM). For metal fracture images classification, experiment shows that fractional cosine transform is better than the cosine transform in fracture images feature description, and the correct recognition rate can be achieved 98.8% based on HMM classification mode

A Novel Pedestrian Dead Reckoning Solution Using Motion Recognition Algorithm with Wearable EMG Sensors

Navigation applications and location-based services are currently becoming standard features in smart phones with built-in GPS receivers. However, a ubiquitous navigation solution which locates a mobile user anytime anywhere is still not available, especially in Global Navigation Satellite System (GNSS) degraded and denied environments. Different motion sensors and angular sensors have been adopted for augmenting the positioning solutions for such environments. An electromyography (EMG) sensor, which measures electrical potentials generated by muscle contractions from human body, is employed in this paper to detect the muscle activities during human locomotion and captures the human walking dynamics for motion recognition and step detection in a Pedestrian Dead Reckoning (PDR) solution. The work presented in this paper is a consecutive step of our pilot studies in developing a novel and robust PDR solution using wearable EMG sensors. The PDR solution includes standing and walking identification, step detection, stride length estimation, and a position calculation with a heading angular sensor. A situation of standing still is identified from the EMG signals collected from a walking process, which has standing and walking dynamics, via a hidden Markov model classifier fed by sample entropy features. Such pre-classified processing reduces the misdetection rate of step detection. After step detection, two stride length estimation methods are investigated for the PDR solution. Firstly, a linear stride length estimation method based on statistic models is investigated to improve the accuracy of the PDR solution. Secondly, five different walking motions are recognized by a motion recognition algorithm based on some particular EMG features, and a fixed stride length is then set for each walking motion to propagate the position. To validate the effectiveness and practicability of the methods mentioned above, some field tests were conducted by a few testers. The test results indicate that the performance of the proposed PDR solution is comparable to that of a commercial GPS receiver in outdoor test under an open-sky environment.

Offline handwritten Arabic cursive text recognition using Hidden Markov Models and re-ranking

Recognition of handwritten Arabic cursive texts is a complex task due to the similarities between letters under different writing styles. In this paper, a word-based off-line recognition system is proposed, using Hidden Markov Models (HMMs). The method employed involves three stages, namely preprocessing, feature extraction and classification. First, words from input scripts are segmented and normalized. Then, a set of intensity features are extracted from each of the segmented words, which is based on a sliding window moving across each mirrored word image. Meanwhile, structure-like features are also extracted including number of subwords and diacritical marks. Finally, these features are applied in a combined scheme for classification. Intensity features are used to train a HMM classifier, whose results are re-ranked using structure-like features for improved recognition rate. In order to validate the proposed techniques, extensive experiments were carried out using the IFN/ENIT database which contains 32,492 handwritten Arabic words. The proposed algorithm yields superior results of improved accuracy in comparison with several typical methods.

Recognizing Human Daily Activities From Accelerometer Signal

Automated recognition of human daily activities from wearable sensor signals has attracted a great deal of interests in many applications including health care, sports and aged care. In this paper, we presented a Hidden Markov Model (HMM)-based recognition method to recognize six human daily activities from sensor signals collected from a single waist-worn tri-axial accelerometer. All training signals from the same activity class are modeled as generated by a HMM, while a Gaussian Mixture Model (GMM) is used to model the continuous observation for each hidden state. A new test signal is classified to the activity class corresponding to the HMM that can produce the highest likelihood. In order to validate the performance of the proposed method, we collected 420 samples from seven subjects and 72 samples from six new subjects, while performing six daily activities including walking, standing, running, jumping, sitting-down, and falling-down. Two batches of experiments were conducted. The experimental results on the first dataset show that the proposed HMM classifier can learn acceleration signals well with low computational complexity, achieving a very high activity recognition accuracy of 94.8%. In the second experiment, the HMM classifier was evaluated by using new data contained in the second dataset and only two samples were misclassified, which demonstrates that the HMM-based recognition system has a good generalization capability. It can be concluded that the proposed method holds a potential in long-term in-situ assessment of human daily activities under ambulatory environment due to its robustness and computational simplicity.

A Human-Machine Interaction Technique: Hand Gesture Recognition Based on Hidden Markov Models with Trajectory of Hand Motion

We have developed an efficient mechanism for real-time hand gesture recognition based on the trajectory of hand motion and the hidden Markov models classifier. In our system, we divide our gestures into single or both hands, one hand have been defined four basic types of directive gesture such as moving upward, downward, leftward, rightward. Then, two hands have twenty-four kinds of combination gesture. However, we apply the most natural and simple way to define eight kinds gestures in our developed human-machine interaction control system so that the users can easily operate the robot. Experimental results reveal that the face tracking rate is more than 97% in general situations and over 94% when the face suffers from temporal occlusion. The efficiency of system execution is very satisfactory, and we are encouraged to commercialize the robot in the near future.