Phoneme Recognition Rate Research Articles

A Pronunciation Prior Assisted Vowel Reduction Detection Framework with Multi-Stream Attention Method

Vowel reduction is a common pronunciation phenomenon in stress-timed languages like English. Native speakers tend to weaken unstressed vowels into a schwa-like sound. It is an essential factor that makes the accent of language learners sound unnatural. To improve vowel reduction detection in a phoneme recognition framework, we propose an end-to-end vowel reduction detection method that introduces pronunciation prior knowledge as auxiliary information. In particular, we have designed two methods for automatically generating pronunciation prior sequences from reference texts and have implemented a main and auxiliary encoder structure that uses hierarchical attention mechanisms to utilize the pronunciation prior information and acoustic information dynamically. In addition, we also propose a method to realize the feature enhancement after encoding by using the attention mechanism between different streams to obtain expanded multi-streams. Compared with the HMM-DNN hybrid method and the general end-to-end method, the average F1 score of our approach for the two types of vowel reduction detection increased by 8.8% and 6.9%, respectively. The overall phoneme recognition rate increased by 5.8% and 5.0%, respectively. The experimental part further analyzes why the pronunciation prior knowledge auxiliary input is effective and the impact of different pronunciation prior knowledge types on performance.

Acquisition of 500 English Words through a TActile Phonemic Sleeve (TAPS).

Recently, a phonemic-based tactile speech communication system was developed with the goal to transmit speech through the skin for people with hearing impairments and those whose auditory and visual channels are overloaded or compromised. The display, called the TActile Phonemic Sleeve (TAPS), consisted of a 4-by-6 tactor array worn on the dorsal and volar surfaces of the forearm. Earlier work showed that people were able to learn the haptic symbols for 39 English phonemes and reach a mean phoneme recognition rate of 86% correct within one to four hours of training. The current research evaluated the acquisition of up to 500 words using TAPS. A total of 51 participants were trained and tested in three studies with increasing number of phonemes and vocabulary sizes. Individual achievements varied, but the results clearly demonstrate the potential of transmitting any English word using TAPS within a reasonable period of learning. Future work will include increasing the speech transmission rate with TAPS by improving the phonemic codes and reducing the inter-phoneme intervals, addressing the reception of words and sentences composed of strings of tactile phonemes, and assessing the performance of TAPS as a speech communication system for people with severe hearing impairments.

Non-Invasive Silent Phoneme Recognition Using Microwave Signals

Besides the recognition of audible speech, there is currently an increasing interest in the recognition of silent speech, which has a range of novel applications. A major obstacle for a wide spread of silent-speech technology is the lack of measurement methods for speech movements that are convenient, non-invasive, portable, and robust at the same time. Therefore, as an alternative to established methods, we examined to what extent different phonemes can be discriminated from the electromagnetic transmission and reflection properties of the vocal tract. To this end, we attached two Vivaldi antennas on the cheek and below the chin of two subjects. While the subjects produced 25 phonemes in multiple phonetic contexts each, we measured the electromagnetic transmission spectra from one antenna to the other, and the reflection spectra for each antenna (radar), in a frequency band from 2–12 GHz. Two classification methods ( k -nearest neighbors and linear discriminant analysis) were trained to predict the phoneme identity from the spectral data. With linear discriminant analysis, cross-validated phoneme recognition rates of 93% and 85% were achieved for the two subjects. Although these results are speaker- and session-dependent, they suggest that electromagnetic transmission and reflection measurements of the vocal tract have great potential for future silent-speech interfaces.

Arabic phonemes recognition using hybrid LVQ/HMM model for continuous speech recognition

In attempt to increase the rate of Arabic phonemes recognition, we introduce a novel hybrid recognition algorithm. The algorithm is composed of the learning vector quantization (LVQ) and hidden Markov model (HMM). The hybrid algorithm used to recognizing Arabic phonemes in continuous open-vocabulary speech. A recorded Arabic corpus of different TV news for modern standard Arabic was used for training and testing purposes. We employ a data driven approach to generate the training feature vectors that embed the frame neighboring correlation information. Next, we generate the phonemes codebooks using the K-means splitting algorithm. Then, we trained the generated codebooks using the LVQ algorithm. We achieved a performance of 98.49 % during independent classification training and 90 % during dependent classification training. When using the trained LVQ codebooks in Arabic utterance transcription, the phoneme recognition rate was 72 % using LVQ only. We combined the LVQ codebooks with the single state HMM model using enhanced Viterbi algorithm which includes the phonemes bigrams. We achieved 89 % of Arabic phonemes recognition rate based on the hybrid LVQ/HMM algorithm.

An improved feature transformation method using mutual information

The feature transformation is a very important step in pattern recognition systems. A feature transformation matrix can be obtained using different criteria such as discrimination between classes or feature independence or mutual information between features and classes. The obtained matrix can also be used for feature reduction. In this paper, we propose a new method for finding a feature transformation-based on Mutual Information (MI). For this purpose, we suppose that the Probability Density Function (PDF) of features in classes is Gaussian, and then we use the gradient ascent to maximize the mutual information between features and classes. Experimental results show that the proposed MI projection consistently outperforms other methods for a variety of cases. In the UCI Glass database we improve the classification accuracy up to 7.95 %. Besides, the improvement of phoneme recognition rate is 3.55 % on TIMIT.

Adjustment Method between Phonological Attributes and Phone Boundaries

Two kinds of imperfections, namely the detection errors and the asynchrony between phonological attributes and phone boundaries, can cause a substantial decline in recognition accuracy of a detection-based automatic speech recognition system. To solve these problems, an adjustment method between phonological attributes and phone boundaries is proposed in this paper. At first the prior knowledge of corpus and the detection results are combined, then the asynchronies in the phone boundary area are compensated and the detection errors are corrected; additionally, by selectively deleting some frames with errors, the precision of the phone models are improved. After adoption of this adjustment method, 1.4% of phoneme recognition rate can be improved in the TIMIT phone classification experiments based on Conditional Random Fields.

Implementing KPCA-based speaker adaptation methods with different optimization algorithms in a Persian ASR system

In this paper, two kernel eigenspace-based speaker adaptation methods are implemented using FARSDAT database and their performances are compared with eigenspace-based ones. In the conducted experiments, short lengths of adaptation speech data (2-5 seconds) are used. Experimental results show that 4.5% improvement in phoneme recognition rate is achieved by supervised eigenspace-based methods. Implementing kernel eigenspace-based methods, 0.6% improves the results gained by utilizing eigenspace-based methods in 2 seconds of adaptation data. While, with this amount of data, traditional speaker adaptation methods cannot work efficiently. In addition, in this work, we employ another optimization algorithm instead of usual numerical methods, which is particle swarm optimization (PSO) and its performance in achieving rapid optimization is investigated.

MLP refined posterior features for noise robust phoneme recognition

In this research, we have investigated improvement in the accuracy and robustness of phoneme recognition by refining posterior features extracted from single stream cepstral features. The refinement process is done using Multi Layer Perceptron (MLP) in a cascaded structure. The combination of frame posterior feature vectors, along with the entropy of each frame, as a confidence measure of posterior vectors, in the context window, is used to train a refiner MLP for estimating a new phoneme posterior feature set with the advantage of more accuracy and robustness. The confidence measure, as an informative feature, would enhance the refiner MLP performance in the correction of misclassified frames. The refiner MLP also models language level phonetics and lexical knowledge, using embedded information in the phoneme posteriors of a large context window. The suitability of these refined posterior features is evaluated on the tandem connectionist structures. Results show a significant improvement in both frame classification and phoneme recognition rates on the TIMIT acoustic and phonetic corpus compared to standard posterior features.

Evolutionary eigenvoice MLLR speaker adaptation

Abstract This paper considers the problem of rapid and robust speaker adaptation in automatic speech recognition (ASR) systems. We propose an approach using combination of eigenspace-based maximum likelihood linear regression (EMLLR) and evolutionary algorithms. To find the best solution for the coefficients estimation problem, we suggest using genetic algorithm (GA) for rapid speaker adaptation. This is due to the fact that genetic algorithms are not as sensitive as expectation maximization (EM) algorithm to the amount of adaptation data. Experimental results on TIMIT database illustrate that genetic algorithm, using random individuals in first population, leads to up to 1.03% improvement in phoneme recognition rate. Moreover, we show that if the first population contains coefficients initially estimated by maximum likelihood criterion, further improvement can be achieved as well. However, the amount of adaptation data does not have considerable effect on the proposed method.

Effect of speech-intrinsic variations on human and automatic recognition of spoken phonemes

The aim of this study is to quantify the gap between the recognition performance of human listeners and an automatic speech recognition (ASR) system with special focus on intrinsic variations of speech, such as speaking rate and effort, altered pitch, and the presence of dialect and accent. Second, it is investigated if the most common ASR features contain all information required to recognize speech in noisy environments by using resynthesized ASR features in listening experiments. For the phoneme recognition task, the ASR system achieved the human performance level only when the signal-to-noise ratio (SNR) was increased by 15 dB, which is an estimate for the human-machine gap in terms of the SNR. The major part of this gap is attributed to the feature extraction stage, since human listeners achieve comparable recognition scores when the SNR difference between unaltered and resynthesized utterances is 10 dB. Intrinsic variabilities result in strong increases of error rates, both in human speech recognition (HSR) and ASR (with a relative increase of up to 120%). An analysis of phoneme duration and recognition rates indicates that human listeners are better able to identify temporal cues than the machine at low SNRs, which suggests incorporating information about the temporal dynamics of speech into ASR systems.

Temporal envelope compensation for robust phoneme recognition using modulation spectrum

A robust feature extraction technique for phoneme recognition is proposed which is based on deriving modulation frequency components from the speech signal. The modulation frequency components are computed from syllable-length segments of sub-band temporal envelopes estimated using frequency domain linear prediction. Although the baseline features provide good performance in clean conditions, the performance degrades significantly in noisy conditions. In this paper, a technique for noise compensation is proposed where an estimate of the noise envelope is subtracted from the noisy speech envelope. The noise compensation technique suppresses the effect of additive noise in speech. The robustness of the proposed features is further enhanced by the gain normalization technique. The normalized temporal envelopes are compressed with static (logarithmic) and dynamic (adaptive loops) compression and are converted into modulation frequency features. These features are used in an automatic phoneme recognition task. Experiments are performed in mismatched train/test conditions where the test data are corrupted with various environmental distortions like telephone channel noise, additive noise, and room reverberation. Experiments are also performed on large amounts of real conversational telephone speech. In these experiments, the proposed features show substantial improvements in phoneme recognition rates compared to other speech analysis techniques. Furthermore, the contribution of various processing stages for robust speech signal representation is analyzed.

Ageing Voices: The Effect of Changes in Voice Parameters on ASR Performance

With ageing, human voices undergo several changes which are typically characterized by increased hoarseness and changes in articulation patterns. In this study, we have examined the effect on Automatic Speech Recognition (ASR) and found that the Word Error Rates (WER) on older voices is 10% absolute higher compared to those of adult voices. Subsequently, we compared several voice source parameters including fundamental frequency, jitter, shimmer, harmonicity, and cepstral peak prominence of adult and older males. Several of these parameters show statistically significant difference for the two groups. However, artificially increasing jitter and shimmer measures do not effect the ASR accuracies significantly. Artificially lowering the fundamental frequency degrades the ASR performance marginally but this drop in performance can be overcome to some extent using Vocal Tract Length Normalisation (VTLN). Overall, we observe that the changes in the voice source parameters do not have a significant impact on ASR performance. Comparison of the likelihood scores of all the phonemes for the two age groups show that there is a systematic mismatch in the acoustic space of the two age groups. Comparison of the phoneme recognition rates show that mid vowels, nasals, and phonemes that depend on the ability to create constrictions with tongue tip for articulation are more affected by ageing than other phonemes.

Denoising and recognition using hidden Markov models with observation distributions modeled by hidden Markov trees

Hidden Markov models have been found very useful for a wide range of applications in machine learning and pattern recognition. The wavelet transform has emerged as a new tool for signal and image analysis. Learning models for wavelet coefficients have been mainly based on fixed-length sequences, but real applications often require to model variable-length, very long or real-time sequences. In this paper, we propose a new learning architecture for sequences analyzed on short-term basis, but not assuming stationarity within each frame. Long-term dependencies will be modeled with a hidden Markov model which, in each internal state, will deal with the local dynamics in the wavelet domain, using a hidden Markov tree. The training algorithms for all the parameters in the composite model are developed using the expectation-maximization framework. This novel learning architecture could be useful for a wide range of applications. We detail two experiments with artificial and real data: model-based denoising and speech recognition. Denoising results indicate that the proposed model and learning algorithm are more effective than previous approaches based on isolated hidden Markov trees. In the case of the ‘Doppler’ benchmark sequence, with 1024 samples and additive white noise, the new method reduced the mean squared error from 1.0 to 0.0842. The proposed methods for feature extraction, modeling and learning, increased the phoneme recognition rates in 28.13%, with better convergence than models based on Gaussian mixtures.

Microscopic prediction of speech recognition for listeners with normal hearing in noise using an auditory model

This study compares the phoneme recognition performance in speech-shaped noise of a microscopic model for speech recognition with the performance of normal-hearing listeners. "Microscopic" is defined in terms of this model twofold. First, the speech recognition rate is predicted on a phoneme-by-phoneme basis. Second, microscopic modeling means that the signal waveforms to be recognized are processed by mimicking elementary parts of human's auditory processing. The model is based on an approach by Holube and Kollmeier [J. Acoust. Soc. Am. 100, 1703-1716 (1996)] and consists of a psychoacoustically and physiologically motivated preprocessing and a simple dynamic-time-warp speech recognizer. The model is evaluated while presenting nonsense speech in a closed-set paradigm. Averaged phoneme recognition rates, specific phoneme recognition rates, and phoneme confusions are analyzed. The influence of different perceptual distance measures and of the model's a-priori knowledge is investigated. The results show that human performance can be predicted by this model using an optimal detector, i.e., identical speech waveforms for both training of the recognizer and testing. The best model performance is yielded by distance measures which focus mainly on small perceptual distances and neglect outliers.

Improving Throat Microphone Speech Recognition by Joint Analysis of Throat and Acoustic Microphone Recordings

We present a new framework for joint analysis of throat and acoustic microphone (TAM) recordings to improve throat microphone only speech recognition. The proposed analysis framework aims to learn joint sub-phone patterns of throat and acoustic microphone recordings through a parallel branch HMM structure. The joint sub-phone patterns define temporally correlated neighborhoods, in which a linear prediction filter estimates a spectrally rich acoustic feature vector from throat feature vectors. Multimodal speech recognition with throat and throat-driven acoustic features significantly improves throat-only speech recognition performance. Experimental evaluations on a parallel TAM database yield benchmark phoneme recognition rates for throat-only and multimodal TAM speech recognition systems as 46.81% and 60.69%, respectively. The proposed throat-driven multimodal speech recognition system improves phoneme recognition rate to 52.58%, a significant relative improvement with respect to the throat-only speech recognition benchmark system.

Modulation frequency features for phoneme recognition in noisy speech

In this letter, a new feature extraction technique based on modulation spectrum derived from syllable-length segments of subband temporal envelopes is proposed. These subband envelopes are derived from autoregressive modeling of Hilbert envelopes of the signal in critical bands, processed by both a static (logarithmic) and a dynamic (adaptive loops) compression. These features are then used for machine recognition of phonemes in telephone speech. Without degrading the performance in clean conditions, the proposed features show significant improvements compared to other state-of-the-art speech analysis techniques. In addition to the overall phoneme recognition rates, the performance with broad phonetic classes is reported.

Use of associative information to continuous speech recognition

AbstractThe continuous speech frequently includes intervals where the utterer is “lazy” and does not exhibit a precise pronunciation. Sometimes the accurate recognition of auxiliaries, which are grammatically important, is prevented. Consequently, to recognize the continuous speech, it is necessary to make use of more intuitive information, i.e., the associative information as the human does, in addition to linguistic information such as syntax and semantics.First, this paper shows that the continuous speech contains ambiguously uttered intervals, for which the acoustic processing is difficult. Next, the human associations among the words are examined and investigated, aiming at the construction of the associative word dictionary to be utilized in the speech recognition. Using such an associative word dictionary, the association strength between words is determined, and the succeeding word is predicted. The associative information, not only the acoustic information, also is included in the recognition of the phrase.In the previous speech recognition system by the authors, the phoneme recognition rate of approximately 90 percent is required in order to achieve the sentence recognition rate of 75 percent, if the associative information is not utilized. The phoneme recognition rate of approximately 80 percent is sufficient to achieve the sentence recognition rate of 75 percent by utilizing the associative information. It is seen also in the system constructed by the authors that the acoustic information and the associative information should be utilized by the ratio of approximately 3:1, for scoring the information.

DICTATION MACHINE BASED ON JAPANESE CHARACTER SOURCE MODELING

This paper describes a phonetic typewriter and a dictation machine that utilize the underlying statistical structure of phoneme or character sequences. The approach of using syllable or character trigrams is applied to language source modeling. The language source models are obtained by calculating trigram probabilities from a large text database. These models are combined with the HMM-LR continuous speech recognition system.3,6 The phonetic typewriter is tested using 274 phrases uttered by one male speaker. The syllable source model achieves a 94.9% phoneme recognition rate with the test-set phoneme perplexity of 3.9. Without the syllable source model, the phoneme recognition rate is only 73.2%. A trigram model based on characters is also evaluated. This character source model can reduce the syllable perplexity significantly to 7.7, compared with 10.5 of the syllable source model. The character source model achieves a 78.5% character transcription rate for the 274 phrase utterances. The experimental results show that a syllable source model and a character source model are very effective for realizing a Japanese dictation machine.

Predictor codebook for speaker‐independent speech recognition

AbstractThis paper discusses a method to handle the diversified dynamic features of speech by representing the dynamic features of speech by spectrum predictors and constructing the codebook containing predictors as the elements. The effectiveness of the method for speaker‐independent speech recognition is examined. Three kinds of predictor structures, i.e., the forward predictor, the backward predictor and the interpolator, are examined. The predictor codebook is constructed by the predictor quantization procedure, which is a small modification of the LBG algorithm. For the evaluation of the phoneme recognition level, two kinds statistical evaluation quantities and the phoneme recognition rate have been considered. It is seen as a result that the predictor codebook can realize a high phoneme separation capability and the robustness against the speaker variation. By combining the process actually into the phrase recognition system, the performance at the continuous speech recognition level is evaluated. In either case, the codebook with the backward predictor as the elements exhibited the highest performance.

Phoneme HMM evaluation algorithm without phoneme labeling applied to continuous speech hmm evaluation

Phoneme Hidden Markov Model (HMM) are generally evaluated in terms of the phoneme recognition rate by using speech data extracted based on phoneme labels. This paper proposes an evaluation method that does not use phoneme labels for extraction. Consequently, phoneme HMMs can be evaluated even if a speech database without phoneme labeling is used. In this study, concatenation training of the phoneme HMMs is executed using a large-scale speaker-independent continuous-speech database. Evaluation of the HMM phoneme recognition rate which is a function of the number of training speakers, using the proposed evaluation method demonstrates its effectiveness.