Isolated Word Recognition Experiments Research Articles

The orthographic representation of a word’s morphological structure: beneficial and detrimental effect for spellers

In this paper we present a review of the literature on the role of a word’s morphological structure in written language processing, with an emphasis on spelling. First, we describe that many orthographies have opted for a representation of a word’s morphological structure. Second, we discuss experiments that have demonstrated the importance of a word’s morphological structure in reading, both in isolated word recognition experiments (so-called blind morphological decomposition) and in reading for meaning. Third, we discuss experimental findings that the written representation of a word’s morphological structure can have beneficial effects in spelling, already in young children with a good morphological awareness. However, several experiments have also shown that, in some circumstances, the speller’s task of representing morphology in written words creates considerable challenges and causes spelling errors rather than providing assistance. Closer inspection of this dissociation between beneficial and harmful effects reveals that two factors play a crucial role in determining the error risk: (a) the distinction between stems and affixes (i.e., morphological accessibility based on semantic transparency) and (b) the frequency with which a morpheme type in a language (stem, affix) must be retrieved in writing texts (accessibility based on type and token frequency). The review offers a theoretical framework against which the other papers in this special issue can be situated.

ОСОБЛИВОСТІ ВИКОРИСТАННЯ MEL-КЕПСТРАЛЬНИХ КОЕФІЦІЄНТІВ ПРИ РОЗПІЗНАВАННІ МОВЛЕННЯ

One of the main forms of human interaction is speech. Nowadays, there are enough useful programs for speech recognition of people who have certain limitations. These programs translate the text that was spoken aloud into the text, to clearly understand what the person wants to say. The main purpose of speech recognition methods is to obtain information as an input voice signal for further clear translation. Modern methods of speech recognition use an important part - language modeling and acoustic modeling. Nowadays, there are quite a number of methods for translating a voice into a test. The best method of recognition is bas on hidden Markov models. Statistical models are using to display sequences of characters. Therefore, the method based on hidden Markov models is one of the best for solving such problems. The article investigates the methods of speech recognition of people with speech disorders in a short dictionary using mel-keptral coefficients. The proposed method applies the criterion used for unbiased estimation of the logarithmic spectrum to the spectral model represented by MEL coefficients. To solve the nonlinear minimization problem involved in the method, they provide an iterative algorithm whose convergence is guaranteed. Examples of speech analysis and results of an isolated word recognition experiment.

A robust feature extraction approach based on an auditory model for classification of speech and expressiveness

Based on an auditory model, the zero-crossings with maximal Teager energy operator (ZCMT) feature extraction approach was described, and then applied to speech and emotion recognition. Three kinds of experiments were carried out. The first kind consists of isolated word recognition experiments in neutral (non-emotional) speech. The results show that the ZCMT approach effectively improves the recognition accuracy by 3.47% in average compared with the Teager energy operator (TEO). Thus, ZCMT feature can be considered as a noise-robust feature for speech recognition. The second kind consists of mono-lingual emotion recognition experiments by using the Taiyuan University of Technology (TYUT) and the Berlin databases. As the average recognition rate of ZCMT approach is 82.19%, the results indicate that the ZCMT features can characterize speech emotions in an effective way. The third kind consists of cross-lingual experiments with three languages. As the accuracy of ZCMT approach only reduced by 1.45%, the results indicate that the ZCMT features can characterize emotions in a language independent way.

Acoustic Model Adaptation Based on Tensor Analysis of Training Models

We present a tensor analysis of acoustic models comprising various speakers in multiple noise conditions, and its application to the new speaker and environment adaptation for speech recognition. The bases used in adaptation are constructed by decomposing the training models in the state, feature dimension, speaker, and noise spaces using multilinear singular value decomposition. The isolated-word recognition experiment demonstrated the effectiveness of the proposed method, showing better performance than eigenvoice in the babble and factory floor noises for the adaptation data longer than approximately 20 s.

Cued Speech automatic recognition in normal-hearing and deaf subjects

This article discusses the automatic recognition of Cued Speech in French based on hidden Markov models (HMMs). Cued Speech is a visual mode which, by using hand shapes in different positions and in combination with lip patterns of speech, makes all the sounds of a spoken language clearly understandable to deaf people. The aim of Cued Speech is to overcome the problems of lipreading and thus enable deaf children and adults to understand spoken language completely. In the current study, the authors demonstrate that visible gestures are as discriminant as audible orofacial gestures. Phoneme recognition and isolated word recognition experiments have been conducted using data from a normal-hearing cuer. The results obtained were very promising, and the study has been extended by applying the proposed methods to a deaf cuer. The achieved results have not shown any significant differences compared to automatic Cued Speech recognition in a normal-hearing subject. In automatic recognition of Cued Speech, lip shape and gesture recognition are required. Moreover, the integration of the two modalities is of great importance. In this study, lip shape component is fused with hand component to realize Cued Speech recognition. Using concatenative feature fusion and multi-stream HMM decision fusion, vowel recognition, consonant recognition, and isolated word recognition experiments have been conducted. For vowel recognition, an 87.6% vowel accuracy was obtained showing a 61.3% relative improvement compared to the sole use of lip shape parameters. In the case of consonant recognition, a 78.9% accuracy was obtained showing a 56% relative improvement compared to the use of lip shape only. In addition to vowel and consonant recognition, a complete phoneme recognition experiment using concatenated feature vectors and Gaussian mixture model (GMM) discrimination was conducted, obtaining a 74.4% phoneme accuracy. Isolated word recognition experiments in both normal-hearing and deaf subjects were also conducted providing a word accuracy of 94.9% and 89%, respectively. The obtained results were compared with those obtained using audio signal, and comparable accuracies were observed.

New Speaker Adaptation Method Using 2-D PCA

This letter describes a speaker adaptation method based on the two-dimensional PCA of training models. In the method, state and dimension of mean vectors are differentiated, and the covariance matrix is computed dimension-wisely. As a result, the speaker weight can contain different weighting for each dimension of mean vectors. In the isolated-word recognition experiments, the proposed method performed better than both eigenvoice and MLLR, for adaptation data longer than about 15 seconds, due to its more elaborate modeling. The method can also be applied to other PCA-based modeling methods where each training model can be represented as a matrix.

Hybrid Simulated Annealing and Its Application to Optimization of Hidden Markov Models for Visual Speech Recognition

We propose a novel stochastic optimization algorithm, hybrid simulated annealing (SA), to train hidden Markov models (HMMs) for visual speech recognition. In our algorithm, SA is combined with a local optimization operator that substitutes a better solution for the current one to improve the convergence speed and the quality of solutions. We mathematically prove that the sequence of the objective values converges in probability to the global optimum in the algorithm. The algorithm is applied to train HMMs that are used as visual speech recognizers. While the popular training method of HMMs, the expectation-maximization algorithm, achieves only local optima in the parameter space, the proposed method can perform global optimization of the parameters of HMMs and thereby obtain solutions yielding improved recognition performance. The superiority of the proposed algorithm to the conventional ones is demonstrated via isolated word recognition experiments.

Stabilised weighted linear prediction

Weighted linear prediction (WLP) is a method to compute all-pole models of speech by applying temporal weighting of the square of the residual signal. By using short-time energy (STE) as a weighting function, this algorithm was originally proposed as an improved linear predictive (LP) method based on emphasising those samples that fit the underlying speech production model well. The original formulation of WLP, however, did not guarantee stability of all-pole models. Therefore, the current work revisits the concept of WLP by introducing a modified short-time energy function leading always to stable all-pole models. This new method, stabilised weighted linear prediction (SWLP), is shown to yield all-pole models whose general performance can be adjusted by properly choosing the length of the STE window, a parameter denoted by M. The study compares the performances of SWLP, minimum variance distortionless response (MVDR), and conventional LP in spectral modelling of speech corrupted by additive noise. The comparisons were performed by computing, for each method, the logarithmic spectral differences between the all-pole spectra extracted from clean and noisy speech in different segmental signal-to-noise ratio (SNR) categories. The results showed that the proposed SWLP algorithm was the most robust method against zero-mean Gaussian noise and the robustness was largest for SWLP with a small M-value. These findings were corroborated by a small listening test in which the majority of the listeners assessed the quality of impulse-train-excited SWLP filters, extracted from noisy speech, to be perceptually closer to original clean speech than the corresponding all-pole responses computed by MVDR. Finally, SWLP was compared to other short-time spectral estimation methods (FFT, LP, MVDR) in isolated word recognition experiments. Recognition accuracy obtained by SWLP, in comparison to other short-time spectral estimation methods, improved already at moderate segmental SNR values for sounds corrupted by zero-mean Gaussian noise. For realistic factory noise of low pass characteristics, the SWLP method improved the recognition results at segmental SNR levels below 0 dB.

Unified framework for basis-based speaker adaptation using 2-mode analysis

A unified 2-mode analysis method is presented for a speaker adaptation method that uses bases. Different structures of training matrices produce different sets of bases in the same framework. Two forms of training matrices are investigated. Bases obtained from the same training matrix as eigenvoice express the same space as that of eigenvoice, hence producing the same recognition result; the new form of training matrix produces the bases that express a different space but also capture the variation of training speakers. In the isolated word recognition experiment, the speaker-adapted model that uses the new bases reduces the error rates of the speaker-independent model and eigenvoice by about 71 and 26% on average, respectively, for adaptation data longer than 3 s.

Robust Audio-Visual Speech Recognition Based on Late Integration

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> Audio-visual speech recognition (AVSR) using acoustic and visual signals of speech has received attention because of its robustness in noisy environments. In this paper, we present a late integration scheme-based AVSR system whose robustness under various noise conditions is improved by enhancing the performance of the three parts composing the system. First, we improve the performance of the visual subsystem by using the stochastic optimization method for the hidden Markov models as the speech recognizer. Second, we propose a new method of considering dynamic characteristics of speech for improved robustness of the acoustic subsystem. Third, the acoustic and the visual subsystems are effectively integrated to produce final robust recognition results by using neural networks. We demonstrate the performance of the proposed methods via speaker-independent isolated word recognition experiments. The results show that the proposed system improves robustness over the conventional system under various noise conditions without a priori knowledge about the noise contained in the speech. </para>

Adaptive Channel Normalization Based on Infomax Algorithm for Robust Speech Recognition

This paper proposes a new data-driven method for high-pass approaches, which suppresses slow-varying noise components. Conventional high-pass approaches are based on the idea of decorrelating the feature vector sequence, and are trying for adaptability to various conditions. The proposed method is based on temporal local decorrelation using the information-maximization theory for each utterance. This is performed on an utterance-by-utterance basis, which provides an adaptive channel normalization filter for each condition. The performance of the proposed method is evaluated by isolated-word recognition experiments with channel distortion. Experimental results show that the proposed method yields outstanding improvement for channel-distorted speech recognition.

Robust Speech Recognition Using Factorial HMMs for Home Environments

We focus on the problem of speech recognition in the presence of nonstationary sudden noise, which is very likely to happen in home environments. As a model compensation method for this problem, we investigated the use of factorial hidden Markov model (FHMM) architecture developed from a clean-speech hidden Markov model (HMM) and a sudden-noise HMM. While in conventional studies this architecture is defined only for static features of the observation vector, we extended it to dynamic features. In addition, we performed home-environment adaptation of FHMMs to the characteristics of a given house. A database recorded by a personal robot called PaPeRo in home environments was used for the evaluation of the proposed method. Isolated word recognition experiments demonstrated the effectiveness of the proposed method under noisy conditions. Home-dependent word FHMMs (HD-FHMMs) reduced the word error rate by 20.5% compared to that of the clean-speech word HMMs.

Robust In-Car Speech Recognition Based on Nonlinear Multiple Regressions

We address issues for improving handsfree speech recognition performance in different car environments using a single distant microphone. In this paper, we propose a nonlinear multiple-regression-based enhancement method for in-car speech recognition. In order to develop a data-driven in-car recognition system, we develop an effective algorithm for adapting the regression parameters to different driving conditions. We also devise the model compensation scheme by synthesizing the training data using the optimal regression parameters and by selecting the optimal HMM for the test speech. Based on isolated word recognition experiments conducted in 15 real car environments, the proposed adaptive regression approach shows an advantage in average relative word error rate (WER) reductions of 52.5% and 14.8%, compared to original noisy speech and ETSI advanced front end, respectively.

Single-Channel Multiple Regression for In-Car Speech Enhancement

We address issues for improving hands-free speech enhancement and speech recognition performance in different car environments using a single distant microphone. This paper describes a new single-channel in-car speech enhancement method that estimates the log spectra of speech at a close-talking microphone based on the nonlinear regression of the log spectra of noisy signal captured by a distant microphone and the estimated noise. The proposed method provides significant overall quality improvements in our subjective evaluation on the regression-enhanced speech, and performed best in most objective measures. Based on our isolated word recognition experiments conducted under 15 real car environments, the proposed adaptive nonlinear regression approach shows an advantage in average relative word error rate (WER) reductions of 50.8% and 13.1%, respectively, compared to original noisy speech and ETSI advanced front-end (ETSI ES 202 050).

Adaptation method using expectation-maximisation for noisy speech recognition

An expectation-maximisation-based adaptation method for the mean of noise is proposed for noisy speech recognition. The noisy testing speech is used for the unsupervised adaptation by employing the parallel model combination (PMC). The proposed method has shown improved performances compared with the PMC method in the isolated word recognition experiments.

On the temporal decorrelation of feature parameters for noise-robust speech recognition

We propose a new frame decorrelation method for robust speech recognition in noisy environments. In most cases, signal perturbation is caused by channel distortion and additive background noise, and can be modeled as a slowly varying term in either the log spectral or the linear-spectral domains. Thus, it is effective to deemphasize slowly varying stationary components in the spectral feature domain of speech signals, which can be considered as a temporal decorrelation process. The proposed method presents a well structured high-pass filter using the decorrelation principle, and provides some significant insights into existing high-pass approaches, such as relative spectral (RASTA) processing. The performance of the proposed method was evaluated by speaker-independent isolated-word recognition experiments using the hidden Markov model (HMM). Noisy speech was simulated by adding noise sources taken from the Noisex-92 database. Experimental results showed that the proposed method was effective for speech recognition with significant noise and yielded better performance than other high-pass methods. In addition, we compared the dynamic property of the proposed filter with that of delta features. The feature obtained by the proposed method may offer most of the delta feature property.

A model of auditory perception as front end for automatic speech recognition.

A front end for automatic speech recognizers is proposed and evaluated which is based on a quantitative model of the "effective" peripheral auditory processing. The model simulates both spectral and temporal properties of sound processing in the auditory system which were found in psychoacoustical and physiological experiments. The robustness of the auditory-based representation of speech was evaluated in speaker-independent, isolated word recognition experiments in different types of additive noise. The results show a higher robustness of the auditory front end in noise, compared to common mel-scale cepstral feature extraction. In a second set of experiments, different processing stages of the auditory front end were modified to study their contribution to robust speech signal representation in detail. The adaptive compression stage which enhances temporal changes of the input signal appeared to be the most important processing stage towards robust speech representation in noise. Low-pass filtering of the fast fluctuating envelope in each frequency band further reduces the influence of noise in the auditory-based representation of speech.

Cepstrum third-order normalisation method for noisy speech recognition

A new cepstrum normalisation method is proposed which can be used to compensate for distortion caused by additive noise. Conventional methods only compensate for the deviation of the cepstral mean and/or variance. However, deviations of higher order moments also exist in noisy speech signals. The proposed method normalises the cepstrum up to its third-order moment, providing closer probability density functions between clean and noisy cepstra than is possible using conventional methods. From the speaker-independent isolated-word recognition experiments, it is shown that the proposed method gives improved performance compared with that of conventional methods, especially in heavy noise environments.

Training section tolerant HMM in concatenated training

Concatenated training of phoneme HMMs can use speech data without hand labels for training HMMs, but it has a tendency to decrease the recognition rate because of an improper training section. The results for speaker-independent isolated word recognition experiments showed that non left–right HMM is tolerant of the quality of the training section in concatenated training than the conventional left-to-right HMM. Non left–right HMM has a structure where state transitions within a phoneme are ergodic and state transitions between successive two phonemes are left-to-right. Non left-right HMM and the conventional left-to-right HMM show much the same performance as long as the training section is given properly by hand labels. Since the training section contains undesirable data, the recognition rate for both HMMs decreases, but the decrement of the recognition rate for non left-right HMM is smaller than the decrement for the conventional left-to-right HMM. Similar tendency was shown when the training section was determined by a phoneme duration model independently from the hand labels.

Feature space reduction through nonlinear identity mapping

Reducing the dimension of acoustic feature space is realized using a wine-glass-type neural network, which has fewer number units in the middle layer than in the input and output layers, trained for the identity mapping. Three-layer wine-glass-type neural networks, which have 32 units for both input and output layers and two to five units for the middle layer are trained so as to map the input 32 LPC derived log spectrum vectors to the identical output vectors. After 500 iterations of backpropagation learning, the network can map the input to itself by 2.7 dB (five units) to 4.14 dB (two units) signal-to-deviation ratio of log spectrum for the training data. For the evaluation, DTW isolated word recognition experiments are performed using 132 similar city name utterances of a male speaker. Using the output of the middle layer units as the reduced feature vector, the recognition accuracy are 75.6% (two units) to 88.6% (five units). Since the accuracy using 16 cepstral coefficients derived from the original speech is 89.4%, the effectiveness of nonlinear identity mapping for reducing the feature dimension is confirmed.