Gaussian Mixture Hidden Markov Models Research Articles

Automatic evaluation method for vehicle audio warning system using MFCC-polynomial hybrid feature

In the evaluation of vehicle audio warning system, there is no automatic method. Besides, due to the noise interference of in-vehicle environmental, the quantity limitation and only positive training samples, the accuracy of traditional template matching or identification methods for audio is low. To solve the above problems, an efficient, accurate, and automatic evaluation method is proposed for vehicle audio warning system. Firstly, logmmse-spectrum subtraction method is used to filter the dynamic noise and static noise of the evaluation audio acquired in the in-vehicle environment. Secondly, the end point detection based on short-time energy is used to obtain the effective audio segment after noise reduction, and the start time of the audio warning segment can be accurately obtained. Then, the Mel Frequency Cepstrum Coefficient (MFCC) feature and the polynomial fitting feature of each audio segment are extracted. The hybrid features are treated as the input of the Hidden Markov Model-Gaussian Mixture Model (GMM-HMM) based audio matching model. Finally, according to frame shift set by endpoint detection and the audio sampling frequency, the emitted time of matched audio warning can be calculated to support the evaluation of vehicle audio warning system. The experimental result shows that, with dynamic-static noise reduction and MFCC-polynomial hybrid feature, the average matching accuracy of the proposed method reaches 99.6% in the case of only five training samples.

Multi-Stage Network Attack Detection Algorithm Based on Gaussian Mixture Hidden Markov Model and Transfer Learning

Multi-Stage Network Attack Detection Algorithm Based on Gaussian Mixture Hidden Markov Model and Transfer Learning

Regular Vehicle Spatial Distribution Estimation Based on Machine Learning

For the mixed traffic flow, obtaining the distribution of connected vehicles (CVs) and regular vehicles (RVs) is of great significance for road network analysis and cooperative control in intelligent transportation systems (ITSs). However, whether it is based on fixed sensors or based on CVs and traffic mechanism to estimate the spatial distribution of RVs, the implementation complexity and low estimation accuracy are the points that need to be improved. This paper proposes a regular vehicle spatial distribution estimation method using adjacent connected vehicles as mobile sensors. First, to investigate the hidden relationship between the interaction information of adjacent CVs and the spatial distribution of RVs among CVs, the Gaussian mixture model-hidden Markov model (GMM-HMM) is selected as the identification method. Then, three sets of experiments were designed to study the influence of observed features on the identification capability of the model, generalization capability validation, and comparison with other methods, respectively. Finally, the proposed method is verified by the dataset generated by the car-following model. The experimental results show that selecting the relative position and time headway as observed features can effectively reflect the regular vehicle spatial distribution between adjacent CVs. The average accuracy of the proposed method to identify the regular vehicle spatial distribution is over 93.7%, which can provide valuable suggestions for the Internet of Vehicles application.

Stock price prediction using a novel approach in Gaussian mixture model-hidden Markov model

PurposeThis study aims to provide the best estimate of a stock's next day's closing price for a given day with the help of the hidden Markov model–Gaussian mixture model (HMM-GMM). The results were compared with Hassan and Nath’s (2005) study using HMM and artificial neural network (ANN).Design/methodology/approachThe study adopted an initialization approach wherein the hidden states of the HMM are modelled as GMM using two different approaches. Training of the HMM-GMM model is carried out using two methods. The prediction was performed by taking the closest closing price (having a log-likelihood within the tolerance range) to that of the present one as the closing price for the next day. Mean absolute percentage error (MAPE) has been used to compare the proposed GMM-HMM model against the models of the research study (Hassan and Nath, 2005).FindingsComparing this study with Hassan and Nath (2005) reveals that the proposed model outperformed in 66 out of the 72 different test cases. The results affirm that the model can be used for more accurate time series prediction. Further, compared with the results of the ANN model from Hassan's study, the proposed HMM model outperformed 24 of the 36 test cases.Originality/valueThe study introduced a novel initialization and two training/prediction approaches for the HMM-GMM model. It is to be noted that the study has introduced a GMM-HMM-based closing price estimator for stock price prediction. The proposed method of forecasting the stock prices using GMM-HMM is explainable and has a solid statistical foundation.

An Efficient Method for the Reliability Evaluation of Power Systems Considering the Variable Photovoltaic Power Output

The operational reliability of power systems is threatened by the random failure of components and uncertain power output of renewable energies, such as photovoltaics. Under such circumstances, reliability evaluation is necessary for maintaining a continuous and stable energy supply. However, traditional reliability evaluation methods are usually extremely time-consuming, considering the numerous system states that need to be analysed. Hence, the reliability evaluation process cannot follow up the dynamic changes in PV output, which makes the timeline of the evaluation disappointing. This paper proposes an efficient reliability evaluation method for power systems with PV integration. The method reveals the analytical relationship between the reliability levels of the power system and the uncertainty factors that influence the reliability, such as the PV output. In this way, the dynamic reliability evaluation is achieved, and the evaluation results can be updated timely when the output of PV changes. First, a Gaussian mixture-hidden Markov model (GMM-HMM) is used to model the distribution characteristics of PV output. Then, the state enumeration and the hyperbolic truncated polynomial chaos expansion method are used to determine the analytical relationship between the reliability indices and PV output. Lastly, based on the analytical function, the operational reliability of the power systems is dynamically evaluated considering the real-time PV output. The effectiveness of the proposed method is verified using the modified IEEE 30 system as an example.

A hierarchical dynamic Bayesian learning network for EMG-based early prediction of voluntary movement intention

Decoding human action intention prior to motion onset with surface electromyograms (sEMG) is an emerging neuroengineering topic with interesting clinical applications such as intelligent control of powered prosthesis/exoskeleton devices. Despite extensive prior works in the related fields, it remains a technical challenge due to considerable variability of complex multi-muscle activation patterns in terms of volatile spatio-temporal characteristics. To address this issue, we first hypothesize that the inherent variability of the idle state immediately preceding the motion initiation needs to be addressed explicitly. We therefore design a hierarchical dynamic Bayesian learning network model that integrates an array of Gaussian mixture model – hidden Markov models (GMM-HMMs), where each GMM-HMM learns the multi-sEMG processes either during the idle state, or during the motion initiation phase of a particular motion task. To test the hypothesis and evaluate the new learning network, we design and build a upper-limb sEMG-joystick motion study system, and collect data from 11 healthy volunteers. The data collection protocol adapted from the psychomotor vigilance task includes repeated and randomized binary hand motion tasks (push or pull) starting from either of two designated idle states: relaxed (with minimal muscle tones), or prepared (with muscle tones). We run a series of cross-validation tests to examine the performance of the method in comparison with the conventional techniques. The results suggest that the idle state recognition favors the dynamic Bayesian model over a static classification model. The results also show a statistically significant improvement in motion prediction accuracy by the proposed method (93.83±6.41%) in comparison with the conventional GMM-HMM method (89.71±8.98%) that does not explicitly account for the idle state. Moreover, we examine the progress of prediction accuracy over the course of motion initiation and identify the important hidden states that warrant future research.

Acoustic Model with Multiple Lexicon Types for Indonesian Speech Recognition

Currently, speech recognition datasets are increasingly available freely in various languages. However, speech recognition datasets in the Indonesian language are still challenging to obtain. Consequently, research focusing on speech recognition is challenging to carry out. This research creates Indonesian speech recognition datasets from YouTube channels with subtitles by validating all utterances of downloaded audio to improve the data quality. The quality of the dataset was evaluated using a deep neural network. The time delay neural network (TDNN) was used to build the acoustic model by applying the alignment data from the Gaussian mixture model-hidden Markov model (GMM-HMM). Data augmentation was used to increase the number of validated datasets and enhance the performance of the acoustic model. The results show that the acoustic model built using the validated datasets is better than the unvalidated datasets for all types of lexicons. Utilizing the four lexicon types and increasing the data through augmentation to train the acoustic models can lower the word error rate percentage in the GMM-HMM, TDNN factorization (TDNNF), and CNN-TDNNF-augmented models to 40.85%, 24.96%, and 19.03%, respectively.

China's GDP forecasting using Long Short Term Memory Recurrent Neural Network and Hidden Markov Model.

This paper presents a Long Short Term Memory Recurrent Neural Network and Hidden Markov Model (LSTM-HMM) to predict China's Gross Domestic Product (GDP) fluctuation state within a rolling time window. We compare the predictive power of LSTM-HMM with other dynamic forecast systems within different time windows, which involves the Hidden Markov Model (HMM), Gaussian Mixture Model-Hidden Markov Model (GMM-HMM) and LSTM-HMM with an input of monthly Consumer Price Index (CPI) or quarterly CPI within 4-year, 6-year, 8-year and 10-year time window. These forecasting models employed in our empirical analysis share the basic HMM structure but differ in the generation of observable CPI fluctuation states. Our forecasting results suggest that (1) among all the models, LSTM-HMM generally performs better than the other models; (2) the model performance can be improved when model input transforms from quarterly to monthly; (3) among all the time windows, models within 10-year time window have better overall performance; (4) within 10-year time window, the LSTM-HMM, with either quarterly or monthly input, has the best accuracy and consistency.

An E2E-ASR-Based Iteratively-Trained Timestamp Estimator

Text-to-speech alignment, also known as time alignment, is essential for automatic speech recognition (ASR) systems used for speech retrieval tasks, such as keyword search and speech segment extraction. Previous works have used the Gaussian mixture model-hidden Markov model (GMM-HMM) forced alignment to improve the alignment performance. However, when used with end-to-end (E2E) ASR, GMM-HMM forced alignment causes extra reliance on expertise such as pronunciation lexica. It also increases the system complexity because GMM-HMMs are very dissimilar to E2E models. To tackle these two problems, we propose an E2E-ASR-based iteratively-trained timestamp estimator (ITSE), which performs alignment between token-level transcription and speech. We train ITSE first with coarse initial alignment targets generated using connectionist temporal classification (CTC) posteriors. During training, we iteratively perform realignment to update the targets. We attribute the effectiveness of the iterative training to ITSE’s two vital features. First, ITSE performs alignment using similarities between token and speech embeddings instead of frame-wise token classification posteriors. Second, ITSE uses speech embeddings that are aware of left context rather than global context. ITSE significantly outperforms CTC-based baselines in word alignment accuracy and is comparable to a GMM-HMM forced aligner. In short, ITSE is an accurate, lightweight text-to-speech alignment module implemented without expertise such as pronunciation lexica.

Sentiment analysis of pets using deep learning technologies in artificial intelligence of things system

This research paper proposes sentiment analysis of pets using deep learning technologies in the artificial intelligence of things system. Mask R-CNN is used to detect image objects and generate contour mask maps, pose analysis algorithms are used to obtain object posture information, and at the same time object sound signals are converted into spectrograms as features, using deep learning image recognition technology to obtain object emotion information. By using the fusion of object posture and emotional characteristics as the basis for pet emotion identification and analysis, the detected specific pet behaviour states will be actively notified to the owner for processing. Compared with traditional speech recognition, which uses mel-frequency cepstral coefficients for feature extraction, coupled with a Gaussian mixture model-hidden Markov model for voice recognition, the experimental method of this research paper effectively improves the accuracy by 70%. Prior work on the implementation of smart pet surveillance systems has used the pet's tail and mouth as features, and has combined these with sound features to analyse the pet's emotions. This research paper proposes a new method of sentiment analysis in pets, and our method is compared with previous related work. Experimental results show that our approach increases the accuracy rate by 70%.

Timestamp-aligning and keyword-biasing end-to-end ASR front-end for a KWS system

Many end-to-end approaches have been proposed to detect predefined keywords. For scenarios of multi-keywords, there are still two bottlenecks that need to be resolved: (1) the distribution of important data that contains keyword(s) is sparse, and (2) the timestamps of the detected keywords are inaccurate. In this paper, to alleviate the first issue and further improve the performance of the end-to-end ASR front-end, we propose the biased loss function for guiding the recognizer to pay more attention to the speech segments containing the predefined keywords. As for the second issue, we solve this problem by modifying the force alignment applied to the end-to-end ASR front-end. To get the frame-level alignment, we utilize a Gaussian Mixture Model-Hidden Markov Model (GMM-HMM) based acoustic model (AM) for auxiliary. The proposed system is evaluated in the OpenSAT20 held by the National Institute of Standards and Technology (NIST). The performance of our end-to-end KWS system is comparable to the conventional hybrid KWS system, sometimes even slightly better. With fusion results of the end-to-end and conventional KWS systems, we won the first prize in the KWS track. On the dev dataset (a part of SAFE-T corpus), the system outperforms the baseline by a large margin, i.e., our system with GMM-HMM aligner has a lower segmentation-aware word error rates (relatively 7.9–19.2% decrease) and higher overall Actual term-weighted values (relatively 3.6–11.0% increase), which demonstrates the effectiveness of the proposed method. For more precise alignments, we can use DNN-based AM as alignmentor at the cost of more computation.

A Gaussian mixture-hidden Markov model of human visual behavior

Vision is an important way for human beings to interact with the outside world and obtain information. In order to research human visual behavior under different conditions, this paper uses a Gaussian mixture-hidden Markov model (GMM-HMM) to model the scanpath, and proposes a new model optimization method, time-shifting segmentation (TSS). The TSS method can highlight the characteristics of the time dimension in the scanpath, improve the pattern recognition results, and enhance the stability of the model. In this paper, a linear discriminant analysis (LDA) method is used for multi-dimensional feature pattern recognition to evaluates the rationality and the accuracy of the proposed model. Four sets of comparative trials were carried out for the model evaluation. The first group applied the GMM-HMM to model the scanpath, and the average accuracy of the classification could reach 0.507, which is greater than the opportunity probability of three classification (0.333). The second set of trial applied TSS method, and the mean accuracy of classification was raised to 0.610. The third group combined GMM-HMM with TSS method, and the mean accuracy of classification reached 0.602, which was more stable than the second model. Finally, comparing the model analysis results with the saccade amplitude (SA) characteristics analysis results, the modeling analysis method is much better than the basic information analysis method. Via analyzing the characteristics of three types of tasks, the results show that the free viewing task have higher specificity value and a higher sensitivity to the cued object search task. In summary, the application of GMM-HMM model has a good performance in scanpath pattern recognition, and the introduction of TSS method can enhance the difference of scanpath characteristics. Especially for the recognition of the scanpath of search-type tasks, the model has better advantages. And it also provides a new solution for a single state eye movement sequence.

Data-driven reliability assessment method of Integrated Energy Systems based on probabilistic deep learning and Gaussian mixture Model-Hidden Markov Model

Reliability analysis of IESs (Integrated Energy System) is complicated because of the complexity of system topology and dynamics and different kinds of uncertainties. Reliability is often calculated based on statistic methods, which always focus on historical performances and neglect the importance of their dynamics and structure. To overcome this problem, in this paper, a systematic framework for dynamically analysing the real-time reliability of IESs is proposed by integrating different machine learning methods and statistics. Firstly, the bootstrap-based Extreme Learning Machine is developed to forecast the conditional probability distributions of the productions of renewable energies and the energy consumptions. Then, the dynamic behaviour of IESs is simulated based on a stacked auto-encoder model, instead of using traditional mechanism-based simulation models, for improving computational efficiency. Besides, the variables representing the transient properties of natural gas pipeline networks, such as delivery pressures and flow rates, are taken as the indicators for quantifying the energy supply security in natural gas pipeline networks. The time-dependent relationships among these indicators and their statistic correlations are modelled for improving the effectiveness of the analysis results. Finally, the reliability assessment is performed by estimating the probability distribution of each functional state of the target IES. A case study of a realistic bi-directional IES is carried out to demonstrate the effectiveness of the proposed method. The results show that the method is able to effectively evaluate the reliability of IESs, which can provide useful information for system operation and management.

A COMPARATIVE RESEARCH ON G-HMM AND TSS TECHNOLOGIES FOR EYE MOVEMENT TRACKING ANALYSIS

Eye movement analysis provides a new way for disease screening, quantification and assessment. In order to track and analyze eye movement scanpaths under different conditions, this paper proposed the Gaussian mixture-Hidden Markov Model (G-HMM) modeling the eye movement scanpath during saccade, combing with the Time-Shifting Segmentation (TSS) method for model optimization, and also the Linear Discriminant Analysis (LDA) method was utilized to perform the recognition and evaluation tasks based on the multi-dimensional features. In the experiments, 800 real scene images of eye-movement sequences datasets were used, and the experimental results show that the G-HMM method has high specificity for free searching tasks and high sensitivity for prompt object search tasks, while TSS can strengthen the difference of eye movement characteristics, which is conducive to eye movement pattern recognition, especially for search tasks.

Development of Speech Recognition Systems in Emergency Call Centers

In this paper, various methodologies of acoustic and language models, as well as labeling methods for automatic speech recognition for spoken dialogues in emergency call centers were investigated and comparatively analyzed. Because of the fact that dialogue speech in call centers has specific context and noisy, emotional environments, available speech recognition systems show poor performance. Therefore, in order to accurately recognize dialogue speeches, the main modules of speech recognition systems—language models and acoustic training methodologies—as well as symmetric data labeling approaches have been investigated and analyzed. To find an effective acoustic model for dialogue data, different types of Gaussian Mixture Model/Hidden Markov Model (GMM/HMM) and Deep Neural Network/Hidden Markov Model (DNN/HMM) methodologies were trained and compared. Additionally, effective language models for dialogue systems were defined based on extrinsic and intrinsic methods. Lastly, our suggested data labeling approaches with spelling correction are compared with common labeling methods resulting in outperforming the other methods with a notable percentage. Based on the results of the experiments, we determined that DNN/HMM for an acoustic model, trigram with Kneser–Ney discounting for a language model and using spelling correction before training data for a labeling method are effective configurations for dialogue speech recognition in emergency call centers. It should be noted that this research was conducted with two different types of datasets collected from emergency calls: the Dialogue dataset (27 h), which encapsulates call agents’ speech, and the Summary dataset (53 h), which contains voiced summaries of those dialogues describing emergency cases. Even though the speech taken from the emergency call center is in the Azerbaijani language, which belongs to the Turkic group of languages, our approaches are not tightly connected to specific language features. Hence, it is anticipated that suggested approaches can be applied to the other languages of the same group.

A Hybrid Hidden Markov Model for Pipeline Leakage Detection

In this paper, a deep neural network hidden Markov model (DNN-HMM) is proposed to detect pipeline leakage location. A long pipeline is divided into several sections and the leakage occurs in different section that is defined as different state of hidden Markov model (HMM). The hybrid HMM, i.e., DNN-HMM, consists of a deep neural network (DNN) with multiple layers to exploit the non-linear data. The DNN is initialized by using a deep belief network (DBN). The DBN is a pre-trained model built by stacking top-down restricted Boltzmann machines (RBM) that compute the emission probabilities for the HMM instead of Gaussian mixture model (GMM). Two comparative studies based on different numbers of states using Gaussian mixture model-hidden Markov model (GMM-HMM) and DNN-HMM are performed. The accuracy of the testing performance between detected state sequence and actual state sequence is measured by micro F1 score. The micro F1 score approaches 0.94 for GMM-HMM method and it is close to 0.95 for DNN-HMM method when the pipeline is divided into three sections. In the experiment that divides the pipeline as five sections, the micro F1 score for GMM-HMM is 0.69, while it approaches 0.96 with DNN-HMM method. The results demonstrate that the DNN-HMM can learn a better model of non-linear data and achieve better performance compared to GMM-HMM method.

Dynamic Bayesian Network-based Surface Roughness Accuracy Grade Prediction in Turning

A hybrid DBN model - Discrete and gaussian mixture hidden markov model with the weighted evidence fusion strategy (DGMWEFS), based on Discrete hidden Markov model (DHMM), Mixture of Gaussians hidden Markov model (MoGHMM), and DS evidence theory, is developed for surface roughness accuracy grade prediction. By analyzing the influence of tool vibration on the surface topography, the singular spectrum with wavelet analysis is proposed to extract the fusion feature (Ec). The ρ comparison is developed to overcome defect of the traditional probability comparison. For the multiple outputs of DHMM and MoGHMM, the weighted evidence fusion strategy based on the DS evidence theory is proposed for final decisions. Experiment of turning the workpiece with multi-material and hardness scale shows that, compared with the DHMM (prediction accuracy 85%) and MoGHMM(prediction accuracy 78%), the DGMWEFS proposed estimates the surface roughness with higher accuracy (prediction accuracy 93%).Therefore, the monitoring strategy proposed can be better used for supervising accuracy grade, which can be readily integrated into a computer integrated manufacturing environment.

Device-Free Human Activity Recognition Based on GMM-HMM Using Channel State Information

This paper presents a machine learning method, Gaussian Mixture Hidden Markov Model (GMM-HMM), for device-free activity recognition using WiFi channel state information (CSI). The basic concept of CSI is introduced and signal changes caused by human activity are described, which demonstrates that human activity can be identified using a unique mapping between action and signal variations. The phase difference expanded matrix is built by the mean and standard deviation of phase difference as feature matrix after linear correction and Savitzky-Golay filter is performed on the CSI raw phase information. The GMM-HMM is used for classification as the human activity can be modeled as the Markov process and the complex activity patterns can be fitted by multiple Gaussian density functions, respectively. The proposed system is verified on the self-collected datasets and several factors affecting the recognition accuracy are analyzed. Furthermore, the system has compared with the previous work. High accuracy and robustness in universal scenarios are realized. Experimental results show that the average recognition accuracy of the proposed system is over 97%.

심층신경망 기반의 음성인식을 위한 절충된 특징 정규화 방식*

Feature normalization is a method to reduce the effect of environmental mismatch between the training and test conditions through the normalization of statistical characteristics of acoustic feature parameters. It demonstrates excellent performance improvement in the traditional Gaussian mixture model-hidden Markov model (GMM-HMM)-based speech recognition system. However, in a deep neural network (DNN)-based speech recognition system, minimizing the effects of environmental mismatch does not necessarily lead to the best performance improvement. In this paper, we attribute the cause of this phenomenon to information loss due to excessive feature normalization. We investigate whether there is a feature normalization method that maximizes the speech recognition performance by properly reducing the impact of environmental mismatch, while preserving useful information for training acoustic models. To this end, we introduce the mean and exponentiated variance normalization (MEVN), which is a compromise between the mean normalization (MN) and the mean and variance normalization (MVN), and compare the performance of DNN-based speech recognition system in noisy and reverberant environments according to the degree of variance normalization. Experimental results reveal that a slight performance improvement is obtained with the MEVN over the MN and the MVN, depending on the degree of variance normalization.

DNN-HMM based acoustic model for continuous pig cough sound recognition

To detect the respiratory disease through pig cough sound in the early stage, a novel method based on Deep Neural Networks-Hidden Markov Model (DNN-HMM) was proposed to construct an acoustic model for continuous pig cough sound recognition. Noises in the continuous pig sounds were eliminated by the Wiener algorithm based on wavelet thresholding the multitaper spectrum, and the experimental corpus was obtained from the denoised continuous pig sounds. The 39-dimensional Mel Frequency Cepstral Coefficients (MFCC) extracted from the corpus were considered as feature vectors. Sounds in pig farms were divided into pig coughs, non-pig coughs, and silence segments. In the HMM, the number of hidden states of pig cough, non-pig cough and silence segments were 5, 5 and 3 respectively, and the observation states represented the feature vectors of the continuous pig sound signal. Based on experiments and empirical theory, the DNN model with 3 hidden layers and 100 nodes per layer was used to describe the correspondence between hidden states and observation serials. Through experiments, the context frames of DNN input were set to 5. Under the condition of optimal parameter setting, the traditional acoustic model Gaussian Mixture Model-Hidden Markov Model (GMM-HMM) was compared with DNN-HMM through a 5-fold cross-validation experiment. It was found that the Word Error Rate (WER) of each group in DNN-HMM was lower than that in GMM-HMM, and the average WER was 3.45% lower. At the same time, the best result of the DNN-HMM model was obtained with the lowest WER of 7.54%, and the average WER was 8.03%. The results showed that the method of DNN-HMM based acoustic model for continuous pig cough sound recognition was stable and reliable. Keywords: DNN-HMM, acoustic model, continuous pig coughs, recognition, pig industry DOI: 10.25165/j.ijabe.20201303.4530 Citation: Zhao J, Li X, Liu W H, Gao Y, Lei M G, Tan H Q, et al. DNN-HMM based acoustic model for continuous pig cough sound recognition. Int J Agric & Biol Eng, 2020; 13(3): 186–193.