Features For Speech Recognition Research Articles

Task-Dependent Modulation of Medial Geniculate Body Is Behaviorally Relevant for Speech Recognition

SummaryRecent work has shown that responses in first-order sensory thalamic nuclei are modulated by cortical areas [1–5]. However, the functional role of such corticothalamic modulation and its relevance for human perception is still unclear. Here, we show in two functional magnetic resonance imaging (fMRI) studies that the neuronal response in the first-order auditory thalamus, the medial geniculate body (MGB), is increased when rapidly varying spectrotemporal features of speech sounds are processed, as compared to processing slowly varying spectrotemporal features of the same sounds. The strength of this task-dependent modulation is positively correlated with the speech recognition scores of individual subjects. These results show that task-dependent modulation of the MGB serves the processing of specific features of speech sounds and is behaviorally relevant for speech recognition. Our findings suggest that the first-order auditory thalamus is not simply a nonspecific gatekeeper controlled by attention [6]. Together with studies in nonhuman mammals [4, 5], our findings imply a mechanism in which the first-order auditory thalamus, possibly by corticothalamic modulation, reacts adaptively to features of sensory input.

Variable resolution spectral/temporal features for automatic speech recognition.

Many studies from speech science have shown that the mel frequency scale more closely matches speech perception than the linear frequency scale. Automatic speech recognition engineers have empirically demonstrated that the use of the mel scale results in more accurate speech recognition than that obtainable with features computed with respect to a linear frequency scale. The features most typically used for automatic speech recognition are mel frequency cepstral coefficients (MFCCs), along with delta and acceleration terms that represent the temporal evolution of MFCC over very short time intervals. However, the MFCC features do not encode the better temporal resolution that is possible at higher frequencies with low‐frequency resolution. In this paper, a two‐dimensional feature set is presented that incorporates good frequency resolution and low‐time resolution at low‐frequencies, and low frequency resolution and good time resolution at high frequencies. These features are computed from overlapping block...

Histogram Equalization Utilizing Window-Based Smoothed CDF Estimation for Feature Compensation

In this letter, we propose a new histogram equalization method to compensate for acoustic mismatches mainly caused by corruption of additive noise and channel distortion in speech recognition. The proposed method employs an improved test cumulative distribution function (CDF) by more accurately smoothing the conventional order statistics-based test CDF with the use of window functions for robust feature compensation. Experiments on the AURORA 2 framework confirmed that the proposed method is effective in compensating speech recognition features by reducing the averaged relative error by 13.12% over the order statistics-based conventional histogram equalization method and by 58.02% over the mel-cepstral-based features for the three test sets.

Perceptual features for automatic speech recognition in noisy environments

The performances of two perceptual properties of the peripheral auditory system, synaptic adaptation and two-tone suppression, are compared for automatic speech recognition (ASR) in an additive noise environment. A simple method of synaptic adaptation as determined by psychoacoustic observations was implemented with temporal processing of speech utilizing a zero-crossing auditory model as a pre-processing front end. The concept is similar to RASTA processing, but instead of bandpass filters, a high-pass infinite impulse response (IIR) filter is used. It is shown that rapid synaptic adaptation may be implemented by temporal processing using the zero-crossing algorithm, not otherwise implementable in the spectral domain implementation. The two-tone suppression was implemented in the zero-crossing auditory model using a companding strategy. Recognition performances with the two perceptual features were evaluated on isolated digits (TIDIGITS) corpus using continuous density HMM recognizer in white, factory, babble and Volvo noise. It is observed that synaptic adaptation performs better in stationary white Gaussian noise. In presence of non-stationary non-Gaussian noise, however, no improvements or a degradation is observed. Moreover, a reciprocal effect is observed with two-tone suppression, with better performance in non-Gaussian real-world noise and degradation in stationary white Gaussian noise.

The radar-graphic speech learning system for hearing impaired

Speech learning is an important foundation for literacy ability. In general, language training needs a professional instrument to analyze speech for supporting the pronunciation of hearing impaired. However, non-professional speech spectrum equipment is very expensive and its output is not easy for hearing impaired to understand and learn. The purpose of this research is to implement a radar-graphic displaying system (RDS) to support the speech learning for hearing-impaired people at low cost and better performance. The components of RDS include a computer connects to a microphone as input device to capture the speech features; a neural network is used to extract the features for speech recognition; a radar map displays the voice message on the screen to support the hearing impaired to learn speech. A system performance evaluation of RDS was performed after the system was implemented.

Combined speech enhancement and auditory modelling for robust distributed speech recognition

The performance of automatic speech recognition (ASR) systems in the presence of noise is an area that has attracted a lot of research interest. Additive noise from interfering noise sources, and convolutional noise arising from transmission channel characteristics both contribute to a degradation of performance in ASR systems. This paper addresses the problem of robustness of speech recognition systems in the first of these conditions, namely additive noise. In particular, the paper examines the use of the auditory model of Li et al. [Li, Q., Soong, F.K., Siohan, O., 2000. A high-performance auditory feature for robust speech recognition. In: Proc. 6th Internat. Conf. on Spoken Language Processing (ICSLP), Vol. III. pp. 51–54] as a front-end for a HMM-based speech recognition system. The choice of this particular auditory model is motivated by the results of a previous study by Flynn and Jones [Flynn, R., Jones, E., 2006. A comparative study of auditory-based front-ends for robust speech recognition using the Aurora 2 database. In: Proc. IET Irish Signals and Systems Conf., Dublin, Ireland. pp. 111–116] in which this auditory model was found to exhibit superior performance for the task of robust speech recognition using the Aurora 2 database [Hirsch, H.G., Pearce, D., 2000. The Aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions. In: Proc. ISCA ITRW ASR2000, Paris, France. pp. 181–188]. In the speech recognition system described here, the input speech is pre-processed using an algorithm for speech enhancement. A number of different methods for the enhancement of speech, combined with the auditory front-end of Li et al., are evaluated for the purpose of robust connected digit recognition. The ETSI basic [ETSI ES 201 108 Ver. 1.1.3, 2003. Speech processing, transmission and quality aspects (STQ); distributed speech recognition; front-end feature extraction algorithm; compression algorithms] and advanced [ETSI ES 202 050 Ver. 1.1.5, 2007. Speech processing, transmission and quality aspects (STQ); distributed speech recognition; advanced front-end feature extraction algorithm; compression algorithms] front-ends proposed for DSR are used as a baseline for comparison. In addition to their effects on speech recognition performance, the speech enhancement algorithms are also assessed using perceptual speech quality tests, in order to examine if a correlation exists between perceived speech quality and recognition performance. Results indicate that the combination of speech enhancement pre-processing and the auditory model front-end provides an improvement in recognition performance in noisy conditions over the ETSI front-ends.

A comparison of visual features for audiovisual automatic speech recognition

The use of visual information from speaker's mouth region have shown to improve the performance of automatic speech recognition (ASR) systems. This is particularly important in presence of noise which even in moderate form severely degrades the speech recognition performance of systems using only audio information. Various sets of features extracted from speaker's mouth region have been used to improve upon the performance of an ASR system in such challenging conditions and have met many successes. To the best of authors knowledge, the effect of using these techniques on recognition performance on the basis of phonemes have not been investigated yet. This paper presents a comparison of phoneme recognition performance using visual features extracted from mouth region-of-interest using discrete cosine transform (DCT) and discrete wavelet transform (DWT). New DCT and DWT features have also been extracted and compared with the previously used one. These features were used along with audio features based on Mel frequency cepstral coefficients (MFCC). This work will help in selecting suitable features for different application and identify the limitations of these methods in recognition of individual phonemes.

Constructing Modulation Frequency Domain-Based Features for Robust Speech Recognition

Data-driven temporal filtering approaches based on a specific optimization technique have been shown to be capable of enhancing the discrimination and robustness of speech features in speech recognition. The filters in these approaches are often obtained with the statistics of the features in the temporal domain. In this paper, we derive new data-driven temporal filters that employ the statistics of the modulation spectra of the speech features. Three new temporal filtering approaches are proposed and based on constrained versions of linear discriminant analysis (LDA), principal component analysis (PCA), and minimum class distance (MCD), respectively. It is shown that these proposed temporal filters can effectively improve the speech recognition accuracy in various noise-corrupted environments. In experiments conducted on Test Set A of the Aurora-2 noisy digits database, these new temporal filters, together with cepstral mean and variance normalization (CMVN), provide average relative error reduction rates of over 40% and 27% when compared with baseline Mel frequency cepstral coefficient (MFCC) processing and CMVN alone, respectively.

Bi-Spectral Acoustic Features for Robust Speech Recognition

The extraction of acoustic features for robust speech recognition is very important for improving its performance in realistic environments. The bi-spectrum based on the Fourier transformation of the third-order cumulants expresses the non-Gaussianity and the phase information of the speech signal, showing the dependency between frequency components. In this letter, we propose a method of extracting short-time bispectral acoustic features with averaging features in a single frame. Merged with the conventional Mel frequency cepstral coefficients (MFCC) based on the power spectrum by the principal component analysis (PCA), the proposed features gave a 6.9% relative lower a word error rate in Japanese broadcast news transcription experiments.

A probabilistic framework for landmark detection based on phonetic features for automatic speech recognition

A probabilistic framework for a landmark-based approach to speech recognition is presented for obtaining multiple landmark sequences in continuous speech. The landmark detection module uses as input acoustic parameters (APs) that capture the acoustic correlates of some of the manner-based phonetic features. The landmarks include stop bursts, vowel onsets, syllabic peaks and dips, fricative onsets and offsets, and sonorant consonant onsets and offsets. Binary classifiers of the manner phonetic features-syllabic, sonorant and continuant-are used for probabilistic detection of these landmarks. The probabilistic framework exploits two properties of the acoustic cues of phonetic features-(1) sufficiency of acoustic cues of a phonetic feature for a probabilistic decision on that feature and (2) invariance of the acoustic cues of a phonetic feature with respect to other phonetic features. Probabilistic landmark sequences are constrained using manner class pronunciation models for isolated word recognition with known vocabulary. The performance of the system is compared with (1) the same probabilistic system but with mel-frequency cepstral coefficients (MFCCs), (2) a hidden Markov model (HMM) based system using APs and (3) a HMM based system using MFCCs.

Cepstral Statistics Compensation and Normalization Using Online Pseudo Stereo Codebooks for Robust Speech Recognition in Additive Noise Environments

This paper proposes several cepstral statistics compensation and normalization algorithms which alleviate the effect of additive noise on cepstral features for speech recognition. The algorithms are simple yet efficient noise reduction techniques that use online-constructed pseudo-stereo codebooks to evaluate the statistics in both clean and noisy environments. The process yields transformations for both clean speech cepstra and noise-corrupted speech cepstra, or for noise-corrupted speech cepstra only, so that the statistics of the transformed speech cepstra are similar for both environments. Experimental results show that these codebook-based algorithms can provide significant performance gains compared to results obtained by using conventional utterance-based normalization approaches. The proposed codebook-based cesptral mean and variance normalization (C-CMVN), linear least squares (LLS) and quadratic least squares (QLS) outperform utterance-based CMVN (U-CMVN) by 26.03%, 22.72% and 27.48%, respectively, in relative word error rate reduction for experiments conducted on Test Set A of the Aurora-2 digit database.

Hybrid wavelet based LPC features for Hindi speech recognition

Hybrid features are presented for speech recognition that uses linear prediction in combination with multi-resolution capabilities of wavelet transform. Wavelet-Based Linear Prediction Coefficients (WBLPC) are obtained by applying 3 and 4-level wavelet decomposition and then having linear prediction of each sub-bands to get total 13 features. These features have been tested using a linear discriminant function and Hidden Markov Model (HMM) based classifier for speaker dependent and independent isolated Hindi digits recognition. 3-level WBLPC features gave higher percentage recognition than LPC features while 4-level WBLPC features using HMM gave the highest percentage recognition for both speaker dependent and independent cases.

Digital filter interpolation of decoded LSFs for distributed continuous speech recognition

A digital filter interpolation of decoded line spectral frequencies (LSFs) that significantly outperforms linear interpolation for large vocabulary distributed continuous speech recognition systems is presented. Experiments were conducted using linear predictive coding (LPC) and LSF-derived speech recognition features, CDHMM acoustic models, triphone units and trigram language models for Brazilian Portuguese.

Comparison of Image Transform-Based Features for Visual Speech Recognition in Clean and Corrupted Videos

We present results of a study into the performance of a variety of different image transform-based feature types for speaker-independent visual speech recognition of isolated digits. This includes the first reported use of features extracted using a discrete curvelet transform. The study will show a comparison of some methods for selecting features of each feature type and show the relative benefits of both static and dynamic visual features. The performance of the features will be tested on both clean video data and also video data corrupted in a variety of ways to assess each feature type's robustness to potential real-world conditions. One of the test conditions involves a novel form of video corruption we call jitter which simulates camera and/or head movement during recording.

Temporal Structure Normalization of Speech Feature for Robust Speech Recognition

This letter presents a new feature normalization technique to normalize the temporal structure of speech features. The temporal structure of the features is partially represented by its power spectral density (PSD). We observed that the PSD of the features varies with the corrupting noise and signal-to-noise ratio. To reduce the PSD variation due to noise, we propose to normalize the PSD of features to a reference function by filtering the features. Experimental results on the AURORA-2 task show that the proposed approach when combined with the mean and variance normalization improves the speech recognition accuracy significantly; the system achieves 69.11% relative error rate reduction over the baseline.

SNR-dependent compression of enhanced Mel sub-band energies for compensation of noise effects on MFCC features

The Mel-frequency cepstral coefficients (MFCC) are most widely used features for speech recognition. But, their performance degrades in presence of additive noise. In this paper, we propose a noise compensation method for Mel sub-bands energies as well as MFCC features. This method includes two steps: Mel sub-band spectral subtraction and compression of Mel sub-band energies. In the compression step, we propose a sub-band SNR-dependent compression function. This function replaces logarithm function in conventional MFCC feature extraction. Experimental results show that the proposed method significantly improves performance of MFCC features in noisy conditions. It decreases word error rate about 70% in SNR value of 0 dB for different types of additive noise.

Static and Dynamic Spectral Features: Their Noise Robustness and Optimal Weights for ASR

In this paper, we investigate the relative noise robustness of dynamic and static spectral features in speech recognition. It is found that the dynamic cepstrum is more robust to additive noise than its static counterpart. The results are consistent across different types of noise and over a wide range of noise levels. To exploit this unequal robustness, we propose a simple yet effective strategy of exponentially weighting the likelihoods that are contributed by the static and dynamic features during the decoding process. The optimal weights are discriminatively trained with a small amount of development data. This method is evaluated on two speaker-independent, connected digit databases, one in English (Aurora 2) and the other in Cantonese (CUDIGIT). For various types of noise at different signal-to-noise ratios (SNRs), the average relative word error rate reductions attained with the discriminatively trained weights are 36.6% and 41.9% for Aurora 2 and CUDIGIT, respectively. Noticeable performance improvement can be observed even when there is channel distortion. The proposed approach is appealing to practical applications because 1) noise estimation is not required, 2) model adaptation is not required, 3)only a minor modification of the decoding process is needed, and 4) only a few feature weights need to be trained

Significance of the Modified Group Delay Feature in Speech Recognition

Spectral representation of speech is complete when both the Fourier transform magnitude and phase spectra are specified. In conventional speech recognition systems, features are generally derived from the short-time magnitude spectrum. Although the importance of Fourier transform phase in speech perception has been realized, few attempts have been made to extract features from it. This is primarily because the resonances of the speech signal which manifest as transitions in the phase spectrum are completely masked by the wrapping of the phase spectrum. Hence, an alternative to processing the Fourier transform phase, for extracting speech features, is to process the group delay function which can be directly computed from the speech signal. The group delay function has been used in earlier efforts, to extract pitch and formant information from the speech signal. In all these efforts, no attempt was made to extract features from the speech signal and use them for speech recognition applications. This is primarily because the group delay function fails to capture the short-time spectral structure of speech owing to zeros that are close to the unit circle in the z-plane and also due to pitch periodicity effects. In this paper, the group delay function is modified to overcome these effects. Cepstral features are extracted from the modified group delay function and are called the modified group delay feature (MODGDF). The MODGDF is used for three speech recognition tasks namely, speaker, language, and continuous-speech recognition. Based on the results of feature and performance evaluation, the significance of the MODGDF as a new feature for speech recognition is discussed

Filtered Dynamics and Fractal Dimensions for Noisy Speech Recognition

We explore methods from fractals and dynamical systems theory for robust processing and recognition of noisy speech. A speech signal is embedded in a multidimensional phase-space and is subsequently filtered exploiting aspects of its unfolded dynamics. Invariant measures (fractal dimensions) of the filtered signal are used as features in automatic speech recognition (ASR). We evaluate the new proposed features as well as the previously proposed multiscale fractal dimension via ASR experiments on the Aurora 2 database. The conducted experiments demonstrate relative improved word accuracy for the fractal features, especially at lower signal-to-noise ratio, when they are combined with the mel-frequency cepstral coefficients

A phoneme duration model considering speaking‐rate and linguistic features for speech recognition

In this paper, we proposed a method of phoneme duration modeling for speech recognition. A phoneme with extremely short or long duration often causes a decline of performance of speech recognition. In order to improve performance of recognition, an estimation of phoneme duration determined by various parameters is required. However, there was no usual method of duration modeling for speech recognition considering the influence of both speaking‐rate and linguistic feature (phoneme location in sentence, part‐of‐speech, et al.), which influence phoneme duration strongly. Therefore, we modeled influence of speaking‐rate by two‐dimensional normal distribution of phoneme duration and local average of vowel duration. Each normal distribution is determined by tree‐based clustering with various questions, which include linguistic feature. With an experiment of estimation of phoneme duration by this model, we acquired 20.8% reduction of standard deviation of estimation error. We also used the proposed duration model for rescoring of N‐best hypothesis of speech recognition. With an experiment of rescoring of recognition results for spontaneous speech, we acquired significant reduction of 4.7% in phoneme error rate.