Traditional Hidden Markov Model Research Articles

Noise-Robust Speech Recognition System based on Multimodal Audio-Visual Approach Using Different Deep Learning Classification Techniques

This paper extends an earlier work on designing a speech recognition system based on Hidden Markov Model (HMM) classification technique of using visual modality in addition to audio modality[1]. Improved off traditional HMM-based Automatic Speech Recognition (ASR) accuracy is achieved by implementing a technique using either RNN-based or CNN-based approach. This research is intending to deliver two contributions: The first contribution is the methodology of choosing the visual features by comparing different visual features extraction methods like Discrete Cosine Transform (DCT), blocked DCT, and Histograms of Oriented Gradients with Local Binary Patterns (HOG+LBP), and applying different dimension reduction techniques like Principal Component Analysis (PCA), auto-encoder, Linear Discriminant Analysis (LDA), t-distributed Stochastic Neighbor Embedding (t-SNE) to find the most effective features vector size. Then the obtained visual features are early integrated with the audio features obtained by using Mel Frequency Cepstral Coefficients (MFCCs) and feed the combined audio-visual feature vector to the classification process. The second contribution of this research is the methodology of developing the classification process using deep learning by comparing different Deep Neural Network (DNN) architectures like Bidirectional Long-Short Term Memory (BiLSTM) and Convolution Neural Network (CNN) with the traditional HMM. The proposed model is evaluated on two multi-speakers AV-ASR datasets named AVletters and GRID with different SNR. The model performs speaker-independent experiments in AVlettter dataset and speaker-dependent in GRID dataset.

3D Skeletal Gesture Recognition via Hidden States Exploration.

Temporal dynamics is an open issue for modeling human body gestures. A solution is resorting to the generative models, such as the hidden Markov model (HMM). Nevertheless, most of the work assumes fixed anchors for each hidden state, which make it hard to describe the explicit temporal structure of gestures. Based on the observation that a gesture is a time series with distinctly defined phases, we propose a new formulation to build temporal compositions of gestures by the low-rank matrix decomposition. The only assumption is that the gesture's "hold" phases with static poses are linearly correlated among each other. As such, a gesture sequence could be segmented into temporal states with semantically meaningful and discriminative concepts. Furthermore, different to traditional HMMs which tend to use specific distance metric for clustering and ignore the temporal contextual information when estimating the emission probability, we utilize the long short-term memory to learn probability distributions over states of HMM. The proposed method is validated on multiple challenging datasets. Experiments demonstrate that our approach can effectively work on a wide range of gestures, and achieve state-of-the-art performance.

Human Interaction Recognition Based on Deep Learning and HMM

This paper proposes a recognition method that combines deep learning with traditional hidden Markov model (HMM) with the aim of improving the recognition accuracy of interaction. First, to construct the classification model, the optimized ALexNet convolutional neural network is used to extract the behavior features, followed by the extraction of features that are used to train the long short-term memory (LSTM) network using the Softmax method. Finally, the particle swarm optimization algorithm is used to fuse the classification results with the traditional HMM classification results so that a hybrid classification model is established to obtain the final behavior recognition result. By conducting experiments on the UT-interaction dataset (six types of interaction behavior), the experimental results show that the hybrid model has higher recognition accuracy than other classical methods.

(1610-3210) MAN - MACHINE INTERACTION SYSTEM FOR SUBJECT INDEPENDENT SIGN LANGUAGE RECOGNITION USING FUZZY HIDDEN MARKOV MODEL

Sign language recognition has spawned more and more interest in human–computer interaction society. The major challenge that SLR recognition faces now is developing methods that will scale well with increasing vocabulary size with a limited set of training data for the signer independent application. The automatic SLR based on hidden Markov models (HMMs) is very sensitive to gesture's shape information that makes the accurate parameters of the HMM not capable of characterizing the ambiguous distributions of the observations in gesture's features. This paper presents an extension of the HMMs using interval type-2 fuzzy sets (IT2FSs) to produce interval type-2 fuzzy HMMs to model uncertainties of hypothesis spaces (unknown varieties of parameters of the decision function). The benefit of this enlargement is that it can control both the randomness and fuzziness of traditional HMM mapping. Membership function (MF) of type-2 FS is three-dimensional that provides additional degrees of freedom to evaluate HMM's uncertainties. This system aspires to be a solution to the scalability problem, i.e. has real potential for application on a large vocabulary. Furthermore, it does not rely on the use of data gloves or other means as input devices, and operates in isolated signer-independent modes. Experimental results show that the interval type-2 fuzzy HMM has a comparable performance as that of the fuzzy HMM but is more robust to the gesture variation, while it retains almost the same computational complexity as that of the FHMM.

Fatigue crack fault diagnosis and prognosis based on hidden semi‐Markov model

Aiming at the fault diagnosis and remaining useful life (RUL) prediction of fatigue cracks of a helicopter main gearbox planet carrier, this article proposes a hidden semi-Markov model (HSMM) methodology, which introduces the explicit state durational distribution parameters into the traditional hidden Markov model (HMM), thus overcoming the limitation of exponential distribution in HMM, retaining strong pattern recognition and classification ability, and improving the diagnostic and prognostic accuracy, and the effectiveness of the method was verified through experiments.

A Security Situation Prediction Algorithm Based on HMM in Mobile Network

The increasingly severe network security situation brings unanticipated challenges to mobile networking. Traditional HMM (Hidden Markov Model) based algorithms for predicting the network security are not accurate, and to address this issue, a weighted HMM based algorithm is proposed to predict the security situation of the mobile network. The multiscale entropy is used to address the low speed of data training in mobile network, whereas the parameters of HMM situation transition matrix are also optimized. Moreover, the autocorrelation coefficient can reasonably use the association between the characteristics of the historical data to predict future security situation. Experimental analysis on DARPA2000 shows that the proposed algorithm is highly competitive, with good performance in prediction speed and accuracy when compared to existing design.

Biologically-Inspired Spike-Based Automatic Speech Recognition of Isolated Digits Over a Reproducing Kernel Hilbert Space.

This paper presents a novel real-time dynamic framework for quantifying time-series structure in spoken words using spikes. Audio signals are converted into multi-channel spike trains using a biologically-inspired leaky integrate-and-fire (LIF) spike generator. These spike trains are mapped into a function space of infinite dimension, i.e., a Reproducing Kernel Hilbert Space (RKHS) using point-process kernels, where a state-space model learns the dynamics of the multidimensional spike input using gradient descent learning. This kernelized recurrent system is very parsimonious and achieves the necessary memory depth via feedback of its internal states when trained discriminatively, utilizing the full context of the phoneme sequence. A main advantage of modeling nonlinear dynamics using state-space trajectories in the RKHS is that it imposes no restriction on the relationship between the exogenous input and its internal state. We are free to choose the input representation with an appropriate kernel, and changing the kernel does not impact the system nor the learning algorithm. Moreover, we show that this novel framework can outperform both traditional hidden Markov model (HMM) speech processing as well as neuromorphic implementations based on spiking neural network (SNN), yielding accurate and ultra-low power word spotters. As a proof of concept, we demonstrate its capabilities using the benchmark TI-46 digit corpus for isolated-word automatic speech recognition (ASR) or keyword spotting. Compared to HMM using Mel-frequency cepstral coefficient (MFCC) front-end without time-derivatives, our MFCC-KAARMA offered improved performance. For spike-train front-end, spike-KAARMA also outperformed state-of-the-art SNN solutions. Furthermore, compared to MFCCs, spike trains provided enhanced noise robustness in certain low signal-to-noise ratio (SNR) regime.

Semiparametric hidden Markov model with non-parametric regression

ABSTRACTThe hidden Markov model regression (HMMR) has been popularly used in many fields such as gene expression and activity recognition. However, the traditional HMMR requires the strong linearity assumption for the emission model. In this article, we propose a hidden Markov model with non-parametric regression (HMM-NR), where the mean and variance of emission model are unknown smooth functions. The new semiparametric model might greatly reduce the modeling bias and thus enhance the applicability of the traditional hidden Markov model regression. We propose an estimation procedure for the transition probability matrix and the non-parametric mean and variance functions by combining the ideas of the EM algorithm and the kernel regression. Simulation studies and a real data set application are used to demonstrate the effectiveness of the new estimation procedure.

HMM-based predictive model for enhancing data quality in WSN

ABSTRACTWireless sensor network (WSN) has been widely used in the areas such as health care and industrial monitoring. However, WSN systems still suffer from inevitable problems of communication interference and data failure. In this paper, an improved Hidden Markov Model (HMM) is proposed to enhance the quality of WSN sensor data. This model can be used to recover the missing data and predict the upcoming data in order to improve the data integrity and reliability ultimately. K-means clustering is firstly used to group sensor data series on the basis of different patterns. Next, Particle Swarm Optimization is applied for optimizing HMM parameters, which is enhanced by a hybrid mutation strategy. Experiments on two real data-sets show that the proposed approach can outperform the baseline models (Naïve Bayes, Grey System, BP-Neural Network and Traditional HMM) on precision of both single-step prediction and multiple-step prediction. The results also demonstrate that the proposed approach can improve data quality of WSN significantly. The proposed model can be further extended for time series prediction in other fields.

Deep neural network training for whispered speech recognition using small databases and generative model sampling

State-of-the-art speech recognition solutions currently employ hidden Markov models (HMMs) to capture the time variability in a speech signal and deep neural networks (DNNs) to model the HMM state distributions. It has been shown that DNN–HMM hybrid systems outperform traditional HMM and Gaussian mixture model (GMM) hybrids in many applications. This improvement is mainly attributed to the ability of DNNs to model more complex data structures. However, having sufficient data samples is one key point in training a high accuracy DNN as a discriminative model. This barrier makes DNNs unsuitable for many applications with limited amounts of data. In this study, we introduce a method to produce an excessive amount of pseudo-samples that requires availability of only a small amount of transcribed data from the target domain. In this method, a universal background model (UBM) is trained to capture a parametric estimate of the data distributions. Next, random sampling is used to generate a large amount of pseudo-samples from the UBM. Frame-Shuffling is then applied to smooth the temporal cepstral trajectories in the generated pseudo-sample sequences to better resemble the temporal characteristics of a natural speech signal. Finally, the pseudo-sample sequences are combined with the original training data to train the DNN–HMM acoustic model of a speech recognizer. The proposed method is evaluated on small-sized sets of neutral and whisper data drawn from the UT-Vocal Effort II corpus. It is shown that phoneme error rates (PERs) of a DNN–HMM based speech recognizer are considerably reduced when incorporating the generated pseudo-samples in the training process, with + 79.0 and + 45.6% relative PER improvements for neutral–neutral training/test and whisper–whisper training/test scenarios, respectively.

Dynamic Gesture Recognition based on LeapMotion and HMM-CART Model

This paper focuses on improving the recognition accuracy of dynamic gesture learning, and proposes dynamic gesture track recognition strategy based on HMM-CART model structure. It integrates the modeling ability of the HMM (Hidden-Markov-Model) for time series data and the interpretability of CART (Classification-And-Regression-Tree) for its ability of fast classification and regression, and use it for dynamic gesture recognition. Time series data of movement gestures collected by LeapMotion sensor will be first divided into four channels: finger shape, palm normal vector, palm ball radius and palm displacement vector, and then HMM in HMM Layer will be built for each channel, finally the likelihood probability of model calculated for each sub-sequence of observation should be classified as the input of the CART model in CART Layer to identify the gesture. Experiments show that the accuracy of dynamic gesture recognition method using HMM-CART model is higher than that of traditional single channel HMM.

An alternative-service recommending algorithm based on semantic similarity

With the development of the Internet of Things (IoT), people's lives have become increasingly convenient. It is desirable for smart home (SH) systems to integrate and leverage the enormous information available from IoT. Information can be analyzed to learn user intentions and automatically provide the appropriate services. However, existing service recommendation models typically do not consider the services that are unavailable in a user's living environment. In order to address this problem, we propose a series of semantic models for SH devices. These semantic models can be used to infer user intentions. Based on the models, we proposed a service recommendation probability model and an alternative-service recommending algorithm. The algorithm is devoted to providing appropriate alternative services when the desired service is unavailable. The algorithm has been implemented and achieves accuracy higher than traditional Hidden Markov Model (HMM). The maximum accuracy achieved is 68.3%.

An improved particle swarm-ant colony hybrid algorithm for HMM training

The traditional parameter estimation methods of Hidden Markov Models (HMM) are easy to fall into local optimum, have higher requirements for initial parameter values and might result in over-coupling phenomena. In order to improve the robustness and identification performance of the model, a novel HMM parameter training method based on an improved particle swarm-ant colony algorithm (IPSAA) is presented. First, extremum disturbance is added into particle swarm optimisation algorithm (PSO) and parameters of ant colony algorithm (ACA) such as stimulating factor, volatilisation coefficients and pheromone are all improved adaptively. Second, the fitness function values of particles' history optimal solutions after PSO coarse search are used to adjust the initial pheromone distribution in fine search of ACA. Finally, Baum-Welch algorithm (B-W) is adopted to locally modify the approximate global optimal solution. The new algorithm not only solves the BW dependency on initial values and the trapped local optimum problem, but also makes full use of the global search ability of IPSAA and local development ability of B-W. The experimental results show that the system using the new algorithm is more efficient, more stable, and has better classification performance than that of traditional HMM training algorithm.

A generalized hidden Markov model and its applications in recognition of cutting states

In the traditional hidden Markov model (HMM) for statistical learning and classification, aleatory uncertainty because of randomness and epistemic uncertainty that is due to the lack of knowledge are not differentiated. In this paper, a generalized hidden Markov model (GHMM) in the context of generalized interval probability theory is proposed. The parameters of GHMM are in the form of generalized interval probability, which provides a concise representation for the two kinds of uncertainty simultaneously. The generalized versions of the forward-backward, Viterbi, and Baum-Welch algorithms in GHMM are developed. The proposed algorithms take advantage of the algebraic property in the generalized interval probability. These algorithms provide an efficient approach to train the GHMM. The trained GHMM is used for state recognition with the criterion of maximum log-likelihood. A case study on recognizing the cutting states in manufacturing processes is provided to demonstrate the application of GHMM. The interval forms of the cutting signal are considered as the inputting of GHMM. The cutting states are recognized based on the learning algorithm of GHMM. With the two uncertainty components quantified, the reliability of GHMM in state recognition is superior to that of HMM.

Infrared Human Posture Recognition Method for Monitoring in Smart Homes Based on Hidden Markov Model

Smart homes are the most important sustainability technology of our future. In smart homes, intelligent monitoring is an important component. However, there is currently no effective method for human posture detection for monitoring in smart homes. So, in this paper, we provide an infrared human posture recognition method for monitoring in sustainable smart homes based on a Hidden Markov Model (HMM). We also trained the model parameters. Our model can be used to effectively classify human postures. Compared with the traditional HMM, this paper puts forward a method to solve the problem of human posture recognition. This paper tries to establish a model of training data according to the characteristics of human postures. Accordingly, this complex problem can be decomposed. Thereby, it can reduce computational complexity. In practical applications, it can improve system performance. Through experimentation in a real environment, the model can identify the different body movement postures by observing the human posture sequence, matching identification and classification process. The results show that the proposed method is feasible and effective for human posture recognition. In addition, for human movement target detection, this paper puts forward a human movement target detection method based on a Gaussian mixture model. For human object contour extraction, this paper puts forward a human object contour extraction method based on the Sobel edge detection operator. Here, we have presented an experiment for human posture recognition, and have also examined our cloud-based monitoring system for elderly people using our method. We have used our method in our actual projects, and the experimental results show that our method is feasible and effective.

A heuristic optimization algorithm for HMM based on SA and EM in machinery diagnosis

This paper proposes a novel hidden Markov model (HMM) based on simulated annealing (SA) algorithm and expectation maximization (EM) algorithm for machinery diagnosis. As traditional HMM is sensitive to initial values and EM is easy to trap into a local optimization, SA is combined to improve HMM which can overcome local optimization searching problem. The proposed HMM has strong ability of global convergence, and optimizes the process of parameters estimation. Finally, through a case study, the computation results illustrate this SAEM-HMM has high efficiency and accuracy, which could help machinery diagnosis in practical.

A Continuous Correlated Beta Process Model for Genetic Ancestry in Admixed Populations.

Admixture and recombination create populations and genomes with genetic ancestry from multiple source populations. Analyses of genetic ancestry in admixed populations are relevant for trait and disease mapping, studies of speciation, and conservation efforts. Consequently, many methods have been developed to infer genome-average ancestry and to deconvolute ancestry into continuous local ancestry blocks or tracts within individuals. Current methods for local ancestry inference perform well when admixture occurred recently or hybridization is ongoing, or when admixture occurred in the distant past such that local ancestry blocks have fixed in the admixed population. However, methods to infer local ancestry frequencies in isolated admixed populations still segregating for ancestry do not exist. In the current paper, I develop and test a continuous correlated beta process model to fill this analytical gap. The method explicitly models autocorrelations in ancestry frequencies at the population-level and uses discriminant analysis of SNP windows to take advantage of ancestry blocks within individuals. Analyses of simulated data sets show that the method is generally accurate such that ancestry frequency estimates exhibited low root-mean-square error and were highly correlated with the true values, particularly when large (±10 or ±20) SNP windows were used. Along these lines, the proposed method outperformed post hoc inference of ancestry frequencies from a traditional hidden Markov model (i.e., the linkage model in structure), particularly when admixture occurred more distantly in the past with little on-going gene flow or was followed by natural selection. The reliability and utility of the method was further assessed by analyzing genetic ancestry in an admixed human population (Uyghur) and three populations from a hybrid zone between Mus domesticus and M. musculus. Considerable variation in ancestry frequencies was detected within and among chromosomes in the Uyghur, with a large region of excess French ancestry harboring a gene with a known disease association. Similar variation was detected in the mouse hybrid zone, with notable constancy in regions of excess ancestry among admixed populations. By filling what has been an analytical gap, the proposed method should be a useful tool for many biologists. A computer program (popanc), written in C++, has been developed based on the proposed method and is available on-line at http://sourceforge.net/projects/popanc/.

A hybrid hidden Markov model towards fault detection of rotating components

A robust hybrid hidden Markov model-based fault detection method is proposed to perform multi-state fault classification of rotating components. The approach presented in this paper enhances the performance of the standard hidden Markov model (HMM) for fault detection by performing a series of pre-processing steps. First, the de-noised time-scale signatures are extracted using wavelet packet decomposition of the vibration data. Subsequently, the Teager Kaiser energy operator is employed to demodulate the time-scale components of the raw vibration signatures, following which the condition indicators are calculated. Out of several possible condition indicators, only relevant features are selected using a decision tree. This pre-processing improves the sensitivity of condition indicators under multiple faults. A Gaussian mixing model-based hidden Markov model (HMM) is then employed for fault detection. The proposed hybrid HMM is an improvement over traditional HMM in that it achieves better separation of the feature space leading to more robust state estimation under multiple fault states and measurement noise scenarios. A simulation employing modulated signals and two experimental validation studies are presented to demonstrate the performance of the proposed method.

Evolutionary structure of hidden Markov models for audio-visual Arabic speech recognition

In this paper, we present an Audio-Visual Automatic Speech Recognition System that combines the acoustic and the visual data. The proposed algorithm here, for modelling the multimodal data, is a Hidden Markov Model (HMM) hybridised with the Genetic Algorithm (GA) to determine its optimal structure. This algorithm is combined with the Baum-Welch algorithm, which allows an effective re-estimation of the probabilities of the HMM. Our experiments show the improvement in the performance of the most promising audio-visual system, based on the combination of GA/HMM model compared to the traditional HMM.

Maximum-likelihood normalization of features increases the robustness of neural-based spoken human-computer interaction

Robust acoustic modeling is essential in the development of automatic speech recognition systems applied to spoken human-computer interaction. To this end, traditional hidden Markov models (HMM) may be improved by hybridizing them with artificial neural networks (ANN). Crucially, ANNs require input values that do not compromize their numerical stability. In spite of the relevance feature normalization has on the success of ANNs in real-world applications, the issue is mostly overlooked on the false premize that “any normalization technique will do”. The paper proposes a gradient-ascent, maximum-likelihood algorithm for feature normalization. Relying on mixtures of logistic densities, it ensures ANN-friendly values that are distributed over the (0, 1) interval in a uniform manner. Some nice properties of the approach are discussed. The algorithm is applied to the normalization of acoustic features for a hybrid ANN/HMM speech recognizer. Experiments on real-world continuous speech recognition tasks are presented. The hybrid system turns out to be positively affected by the proposed technique.