HMM-based System Research Articles

Voice Assignment in Vocal Quartets Using Deep Learning Models Based on Pitch Salience

This paper deals with the automatic transcription of four-part, a cappella singing, audio performances. In particular, we exploit an existing, deep-learning based, multiple F0 estimation method and complement it with two neural network architectures for voice assignment (VA) in order to create a music transcription system that converts an input audio mixture into four pitch contours. To train our VA models, we create a novel synthetic dataset by collecting 5381 choral music scores from public-domain music archives, which we make publicly available for further research. We compare the performance of the proposed VA models on different types of input data, as well as to a hidden Markov model-based baseline system. In addition, we assess the generalization capabilities of these models on audio recordings with differing pitch distributions and vocal music styles. Our experiments show that the two proposed models, a CNN and a ConvLSTM, have very similar performance, and both of them outperform the baseline HMM-based system. We also observe a high confusion rate between the alto and tenor voice parts, which commonly have overlapping pitch ranges, while the bass voice has the highest scores in all evaluated scenarios.

Machine Transliteration Using SVM and HMM

Name transliteration plays an important role in developing automatic machine translation and cross lingual information retrieval system because these systems cannot directly translate out-of-vocabulary (OOV) words. In this article, a SVM-based name transliteration approach has been presented. This approach considers transliteration task as a multi-class problem of pattern classification, where the input is a source transliteration unit (chunks of source grapheme) and the classes are the distinct transliteration units (chunks of target grapheme) in the target language. A study on using hidden Markov model (HMM) for solving machine transliteration problem viewed as a sequence learning problem has also been presented in this paper. Bengali-to-English forward and backward name transliteration have been considered in this study. Our proposed methods have been compared with some existing transliteration method that uses a modified version of joint-source channel model. After the systems have been evaluated, the obtained results show that our proposed SVM-based model gives the best results among the others. Our experiments also reveal that the performance of HMM-based system is comparable with the SVM-based system.

Local Key Estimation in Music Recordings: A Case Study Across Songs, Versions, and Annotators

While global key and chord estimation for both popular and classical music recordings have received a lot of attention, little research has been devoted to estimating the local key for classical music. Partly, this may be due to its inherent ambiguity and subjectivity, which makes annotating local keys a challenging task. In this article, we approach local key estimation with a cross-version dataset comprising nine performances (versions) of Schubert's song cycle Winterreise annotated by three different music theory experts. We consider two baseline methods that are representative for common types of signal processing algorithms: an HMM-based system and a CNN-based approach. For both models, we employ a similar training procedure including the optimization of hyperparameters on a validation split. We systematically evaluate the model predictions and provide musical explanations for key confusions. As a central contribution, we explore how different training-test splits affect the models' efficacy. Splitting along the song axis, we find that both methods perform similarly well. Splitting along the version axis, we obtain substantially higher accuracies, especially for the CNN, which seems to effectively learn the harmonic progressions of the songs (“cover song effect”) and successfully generalizes to unseen versions. We further discuss the results for several songs in detail and assess our results from the perspective of multiple annotators. This cross-annotator study reveals that a substantial part of the systems' errors coincides with annotator disagreement and an even larger part can be traced back to musically explainable relationships among different keys.

Facial expression recognition using best tree RD-LGP encoded features and HMM

Hand and face gestures enable deaf people to communicate in their daily lives rather than speaking. This paper describes a hidden Markov model (HMM)-based framework for face sign recognition and detection. The observation vectors used to characterize the states of the HMM are obtained using the best tree local gradient pattern (LGP) encoded features. Each face gesture is modeled as a five-state HMM. The problem of facial expression classification is posed as a composite seven-classes multi-hypothesis Bayesian test. The likelihood ratio test showed that the overall recognition rate for the proposed model is higher than the HMM-local binary pattern descriptor by 6.4%. The overall recognition rate is enhanced by 8.6% using the discrete wavelet packet best tree decomposition filter as a pre-processing noise removal tool. In addition, the overall recognition rate ranges from 84.3%, for the seven classes Bayesian test, to 100%, for lower number of classes depending upon the type of the face gesture. The proposed face expression algorithm reduces significantly the computational complexity of previous HMM-based face expression recognition systems, and still preserve the recognition rate.

Hindi named entity recognition using system combination

This paper reports development of Hindi named entity recognition (NER) system using various machine learning algorithms and their combination. Three machine learning algorithms namely simple K-nearest neighbour (KNN), weighted K-nearest neighbour and HMM have been used for developing three different Hindi NER systems which are finally combined through majority voting to develop an improved hybrid NER system. We have designed set of novel features which have been used for improving the performances of the KNN-based NER systems. A comparative study among our developed NER models has been made based on some standard data sets. This study reveals that performance of the weighted KNN-based NER system is comparable with that of the HMM-based NER system and performance of the hybrid system is significantly better than each individual base model. Our experimental results also show that performance of the hybrid NER system is better than some existing state-of-the art Hindi NER systems.

A segmental HMM based trajectory classification using genetic algorithm

Trajectory classification techniques face various challenges due to varying length and lack of the presence of clear boundaries among the trajectory classes. To overcome such challenges, a trajectory shrinking framework using Adaptive Multi-Kernel based Shrinkage (AMKS) can be used. However, such a strategy often results in over-shrinking of trajectories leading to poor classification. To improve classification performance, we introduce two additional kernels that are based on convex hull and Ramer–Douglas–Peucker (RDP) algorithm. Next, we propose a supervised trajectory classification approach using a combination of global and Segmental Hidden Markov Model (HMM) based classifiers. In the first stage, HMM is used globally for classification of trajectory to provide state-wise distribution of trajectory segments. In the second stage, state-wise trajectory segments are classified and combined with global recognition performance to improve the classification results. Combination of Global HMM and Segmental HMM is performed using a genetic algorithm (GA) based framework in the final stage. We have conducted experiments over two publicly available datasets, popularly known as T15 and MIT. We have achieved 94.80% and 96.75% of accuracies on T15 and MIT datasets, respectively. We also analyzed the robustness of the proposed framework by adding Gaussian noise. To show the effectiveness of the system, we have performed recognition of on-line signature using proposed Segmental HMM based combination model. In SVC2004 signature dataset, it outperforms traditional HMM-based systems.

LSTM Deep Neural Networks Postfiltering for Enhancing Synthetic Voices

Recent developments in speech synthesis have produced systems capable of producing speech which closely resembles natural speech, and researchers now strive to create models that more accurately mimic human voices. One such development is the incorporation of multiple linguistic styles in various languages and accents. Speech synthesis based on Hidden Markov Models (HMM) is of great interest to researchers, due to its ability to produce sophisticated features with a small footprint. Despite some progress, its quality has not yet reached the level of the current predominant unit-selection approaches, which select and concatenate recordings of real speech, and work has been conducted to try to improve HMM-based systems. In this paper, we present an application of long short-term memory (LSTM) deep neural networks as a postfiltering step in HMM-based speech synthesis. Our motivation stems from a similar desire to obtain characteristics which are closer to those of natural speech. The paper analyzes four types of postfilters obtained using five voices, which range from a single postfilter to enhance all the parameters, to a multi-stream proposal which separately enhances groups of parameters. The different proposals are evaluated using three objective measures and are statistically compared to determine any significance between them. The results described in the paper indicate that HMM-based voices can be enhanced using this approach, specially for the multi-stream postfilters on the considered objective measures.

Discriminative keyword spotting using triphones information and N-best search

Keyword Spotting (KWS) systems can be divided into two main groups: Hidden Markov Model (HMM)-based and Discriminative KWS (DKWS) systems. In this paper, we propose an approach to improve a DKWS system using advantages of HMM-based systems. The proposed DKWS system contains feature extraction and classification (that includes a classifier and a search algorithm) parts. The focus of this paper is on the feature extraction part and the search algorithm. At first, we propose a method for using the advantages of a triphone-based HMM system and improving the monophone-based feature extraction, (proposed in our previous works), to triphone-based one. Then, we propose an N-best search algorithm instead of one-best algorithm. The results on TIMIT database indicate that the true detection rate of the triphone-based Evolutionary DKWS (EDKWS) system with N-best search (Tph-EDKWS-N-Best), in false alarm rate per keyword per hour greater than two, is 4.6% higher than that of the monophone-based EDKWS system with one-best search (Mph-EDKWS-1-Best). This improvement costs about 0.4 unit degradation in Real Time Factor (a common metric of measuring the speed of an automatic speech recognition system). Additionally, Figure of Merit (average true detection rate for different false alarm per keyword per hour from 1 to 10) of the Tph-EDKWS-N-Best system is noticeably higher than that of HMM-based KWS systems. However, the computational complexity of the Tph-EDKWS-N-Best system is considerably higher than that of the HMM-based KWS systems.

Video-realistic expressive audio-visual speech synthesis for the Greek language

High quality expressive speech synthesis has been a long-standing goal towards natural human-computer interaction. Generating a talking head which is both realistic and expressive appears to be a considerable challenge, due to both the high complexity in the acoustic and visual streams and the large non-discrete number of emotional states we would like the talking head to be able to express. In order to cover all the desired emotions, a significant amount of data is required, which poses an additional time-consuming data collection challenge. In this paper we attempt to address the aforementioned problems in an audio-visual context. Towards this goal, we propose two deep neural network (DNN) architectures for Video-realistic Expressive Audio-Visual Text-To-Speech synthesis (EAVTTS) and evaluate them by comparing them directly both to traditional hidden Markov model (HMM) based EAVTTS, as well as a concatenative unit selection EAVTTS approach, both on the realism and the expressiveness of the generated talking head. Next, we investigate adaptation and interpolation techniques to address the problem of covering the large emotional space. We use HMM interpolation in order to generate different levels of intensity for an emotion, as well as investigate whether it is possible to generate speech with intermediate speaking styles between two emotions. In addition, we employ HMM adaptation to adapt an HMM-based system to another emotion using only a limited amount of adaptation data from the target emotion. We performed an extensive experimental evaluation on a medium sized audio-visual corpus covering three emotions, namely anger, sadness and happiness, as well as neutral reading style. Our results show that DNN-based models outperform HMMs and unit selection on both the realism and expressiveness of the generated talking heads, while in terms of adaptation we can successfully adapt an audio-visual HMM set trained on a neutral speaking style database to a target emotion. Finally, we show that HMM interpolation can indeed generate different levels of intensity for EAVTTS by interpolating an emotion with the neutral reading style, as well as in some cases, generate audio-visual speech with intermediate expressions between two emotions.

Classifier Architectures for Acoustic Scenes and Events: Implications for DNNs, TDNNs, and Perceptual Features from DCASE 2016

This paper evaluates neural network NN based systems and compares them to Gaussian mixture model GMM and hidden Markov model HMM approaches for acoustic scene classification SC and polyphonic acoustic event detection AED that are applied to data of the “Detection and Classification of Acoustic Scenes and Events 2016” DCASE'16 challenge, task 1 and task 3, respectively. For both tasks, the use of deep neural networks DNNs and features based on an amplitude modulation filterbank and a Gabor filterbank GFB are evaluated and compared to standard approaches. For SC, additionally a time-delay NN approach is proposed that enables analysis of long contextual information similar to recurrent NNs but with training efforts comparable to conventional DNNs. The SC system proposed for task 1 of the DCASE'16 challenge attains a recognition accuracy of 77.5%, which is 5.6% higher compared to the DCASE'16 baseline system. For the AED task, DNNs are adopted in tandem and hybrid approaches, i.e., as part of HMM-based systems. These systems are evaluated for the polyphonic data of task 3 from the DCASE'16 challenge. Several strategies to address the issue of polyphony are considered. It is shown that DNN-based systems perform less accurate than the traditional systems for this task. Best results are achieved using GFB features in combination with a multiclass GMM-HMM back end.

Helpful Statistics in Recognizing Basic Arabic Phonemes

The recognition of continuous speech is one of the main challenges in the building of automatic speech recognition (ASR) systems, especially when it comes to phonetically complex languages such as Arabic. An ASR system seems to be actually in a blocked alley. Nearly all solutions follow the same general model. The previous research focused on enhancing its performance by incorporating supplementary features. This paper is part of ongoing research efforts aimed at developing a high-performance Arabic speech recognition system for learning and teaching purposes. It investigates a statistical analysis of certain distinctive features of the basic Arabic phonemes which seems helpful in enhancing the performance of a baseline HMM-based ASR system. The statistics are collected using a particular Arabic speech database, which involves ten different male speakers and more than eight hours of speech which covers all Arabic phonemes. In HMM modeling framework, the statistics provided are helpful in establishing the appropriate number of HMM states for each phoneme and they can also be utilized as an initial condition for the EM estimation procedure, which generally, accelerates the estimation process and, thus, improves the performance of the system. The obtained findings are presented and possible applications of automatic speech recognition and speaker identification systems are also suggested.

Global Variance in Speech Synthesis With Linear Dynamical Models

Linear Dynamical Models (LDMs) have been used in speech synthesis recently as an alternative to hidden Markov models (HMMs). Among the advantages of LDMs are the ability to capture the dynamics of speech and the achievement of synthesized speech quality similar to HMM-based speech systems on a smaller footprint. However, such as in the HMM case, LDMs produce over-smoothed trajectories of speech parameters, resulting in muffled quality of synthetic speech. Inspired by a similar problem found in HMM-based speech synthesis, where the naturalness of the synthesized speech is greatly improved when the global variance (GV) is compensated, this paper proposes a novel speech parameter generation algorithm that considers GV in LDM-based speech synthesis. Experimental results show that the application of GV during parameter generation significantly improves speech quality.

Unsupervised Word Segmentation and Lexicon Discovery Using Acoustic Word Embeddings

In settings where only unlabelled speech data is available, speech technology needs to be developed without transcriptions, pronunciation dictionaries, or language modelling text. A similar problem is faced when modelling infant language acquisition. In these cases, categorical linguistic structure needs to be discovered directly from speech audio. We present a novel unsupervised Bayesian model that segments unlabelled speech and clusters the segments into hypothesized word groupings. The result is a complete unsupervised tokenization of the input speech in terms of discovered word types. In our approach, a potential word segment (of arbitrary length) is embedded in a fixed-dimensional acoustic vector space. The model, implemented as a Gibbs sampler, then builds a whole-word acoustic model in this space while jointly performing segmentation. We report word error rates in a small-vocabulary connected digit recognition task by mapping the unsupervised decoded output to ground truth transcriptions. The model achieves around 20% error rate, outperforming a previous HMM-based system by about 10% absolute. Moreover, in contrast to the baseline, our model does not require a pre-specified vocabulary size.

An efficient hardware architecture for HMM-based TTS system

This work proposes a hardware architecture for HMM-based text-to-speech synthesis system (HTS). In high speed platforms, HTS with software core-engine can satisfy the requirement of real-time processing. However, in low speed platforms, software core-engine consumes long time-cost to complete the synthesis process. A co-processor was designed and integrated into HTS to accelerate the performance of system.

Acoustic Vowel Analysis in a Mexican Spanish HMM-based Speech Synthesis

The synthetic voice produced from an HMM-based system is often reported as sounding mued when it is compared to natural speech. There are several reasons for this eect: some precise and ne characteristics of the natural speech are removed, minimized or hidden in the modeling phase of the HMM system; the resulting speech-parameter trajectories become oversmoothed versions of the speech waveforms. In order to obtain more natural synthetic voices, dierent training con- ditions must be tried in the construction of the HMMs. One of the most important issues related to the obtained synthetic voice is that of quality assessment. There are several ways to address this, from sub- jective to objective approaches, applied to dierent parameters. This paper presents a comparative analysis of certain acoustic features derived from synthesized speech which has been obtained using dierent training congurations. Pitch, jitter and shimmer were extracted from the syn- thesized versions of three training sets of vowels of a Mexican Spanish speech database: the normal training set and sets with alterations in the context and fundamental frequency F0. The results show that these objective features can be part of an adequate quality assessment of synthetic speech.

A Text Categorization Model Based on Hidden Markov Models

The Hidden Markov model (HMM) has been successfully used for speech recognition, part of speech tagging, and pattern recognition. In this study, we apply the HMM to automatically categorize digital documents into a standard library classification scheme. In the proposed framework, A HMM-based system is viewed as a model to generate a list of words and each document is seen as. . .

Classification of Multiple Seizure-Like States in Three Different Rodent Models of Epileptogenesis

Epilepsy is a dynamical disease and its effects are evident in over fifty million people worldwide. This study focused on objective classification of the multiple states involved in the brain's epileptiform activity. Four datasets from three different rodent hippocampal preparations were explored, wherein seizure-like-events (SLE) were induced by the perfusion of a low - Mg(2+) /high-K(+) solution or 4-Aminopyridine. Local field potentials were recorded from CA3 pyramidal neurons and interneurons and modeled as Markov processes. Specifically, hidden Markov models (HMM) were used to determine the nature of the states present. Properties of the Hilbert transform were used to construct the feature spaces for HMM training. By sequentially applying the HMM training algorithm, multiple states were identified both in episodes of SLE and nonSLE activity. Specifically, preSLE and postSLE states were differentiated and multiple inner SLE states were identified. This was accomplished using features extracted from the lower frequencies (1-4 Hz, 4-8 Hz) alongside those of both the low- (40-100 Hz) and high-gamma (100-200 Hz) of the recorded electrical activity. The learning paradigm of this HMM-based system eliminates the inherent bias associated with other learning algorithms that depend on predetermined state segmentation and renders it an appropriate candidate for SLE classification.

Two-pass search strategy using accumulated band energy histogram for HMM-based identification of perceptually identical music

In this article, we present an efficient two-pass search strategy for the implementation of a Hidden Markov Model (HMM)-based music identification system. In our previous work, we demonstrated a single-pass HMM-based music identification system, considering its application to music copyright protection. This conventional system showed very robust performance to signal-level variations between perceptually identical music files. However, it requires heavy computation for search. In the proposed two-pass search system, the conventional single-pass search is extended to two-pass. In pass 1 of the proposed method, a queried music produces an accumulated band energy histogram which is a set of normalized sums of band energies for each frequency bin. This histogram is compared to all of the histograms for the registered music files. The system generates a list of small number of most probable music files among the all of the registered music files. In pass 2, HMM-based search is applied only to candidate music files selected in pass 1. Using the proposed two-pass strategy, we successfully implemented a HMM-based music identification system, which maintains the same level robustness to signal level variations between perceptually identical music files but also produces the identification result very quickly. © 2013 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 23, 127–132, 2013

A HMM-Based System To Diacritize Arabic Text

The Arabic language comes under the category of Semitic languages with an entirely different sentence structure in terms of Natural Language Processing. In such languages, two different words may have identical spelling whereas their pronunciations and meanings are totally different. To remove this ambiguity, special marks are put above or below the spelling characters to determine the correct pronunciation. These marks are called diacritics and the language that uses them is called a diacritized language. This paper presents a system for Arabic language diacritization using Hid- den Markov Models (HMMs). The system employs the renowned HMM Tool Kit (HTK). Each single diacritic is represented as a separate model. The concatenation of output models is coupled with the input character sequence to form the fully diacritized text. The performance of the proposed system is assessed using a data corpus that includes more than 24000 sentences.

Unsupervised Intralingual and Cross-Lingual Speaker Adaptation for HMM-Based Speech Synthesis Using Two-Pass Decision Tree Construction

Hidden Markov model (HMM)-based speech synthesis systems possess several advantages over concatenative synthesis systems. One such advantage is the relative ease with which HMM-based systems are adapted to speakers not present in the training dataset. Speaker adaptation methods used in the field of HMM-based automatic speech recognition (ASR) are adopted for this task. In the case of unsupervised speaker adaptation, previous work has used a supplementary set of acoustic models to estimate the transcription of the adaptation data. This paper first presents an approach to the unsupervised speaker adaptation task for HMM-based speech synthesis models which avoids the need for such supplementary acoustic models. This is achieved by defining a mapping between HMM-based synthesis models and ASR-style models, via a two-pass decision tree construction process. Second, it is shown that this mapping also enables unsupervised adaptation of HMM-based speech synthesis models without the need to perform linguistic analysis of the estimated transcription of the adaptation data. Third, this paper demonstrates how this technique lends itself to the task of unsupervised cross-lingual adaptation of HMM-based speech synthesis models, and explains the advantages of such an approach. Finally, listener evaluations reveal that the proposed unsupervised adaptation methods deliver performance approaching that of supervised adaptation.