Adaptation For Speech Recognition Research Articles

Adaptation to Noise in Spectrotemporal Modulation Detection and Word Recognition.

Noise adaptation is the improvement in auditory function as the signal of interest is delayed in the noise. Here, we investigated if noise adaptation occurs in spectral, temporal, and spectrotemporal modulation detection as well as in speech recognition. Eighteen normal-hearing adults participated in the experiments. In the modulation detection tasks, the signal was a 200ms spectrally and/or temporally modulated ripple noise. The spectral modulation rate was two cycles per octave, the temporal modulation rate was 10 Hz, and the spectrotemporal modulations combined these two modulations, which resulted in a downward-moving ripple. A control experiment was performed to determine if the results generalized to upward-moving ripples. In the speech recognition task, the signal consisted of disyllabic words unprocessed or vocoded to maintain only envelope cues. Modulation detection thresholds at 0 dB signal-to-noise ratio and speech reception thresholds were measured in quiet and in white noise (at 60 dB SPL) for noise-signal onset delays of 50 ms (early condition) and 800 ms (late condition). Adaptation was calculated as the threshold difference between the early and late conditions. Adaptation in word recognition was statistically significant for vocoded words (2.1 dB) but not for natural words (0.6 dB). Adaptation was found to be statistically significant in spectral (2.1 dB) and temporal (2.2 dB) modulation detection but not in spectrotemporal modulation detection (downward ripple: 0.0 dB, upward ripple: -0.4 dB). Findings suggest that noise adaptation in speech recognition is unrelated to improvements in the encoding of spectrotemporal modulation cues.

Adaptation to Noise in Human Speech Recognition Depends on Noise-Level Statistics and Fast Dynamic-Range Compression.

Human hearing adapts to background noise, as evidenced by the fact that listeners recognize more isolated words when words are presented later rather than earlier in noise. This adaptation likely occurs because the leading noise shifts ("adapts") the dynamic range of auditory neurons, which can improve the neural encoding of speech spectral and temporal cues. Because neural dynamic range adaptation depends on stimulus-level statistics, here we investigated the importance of "statistical" adaptation for improving speech recognition in noisy backgrounds. We compared the recognition of noised-masked words in the presence and in the absence of adapting noise precursors whose level was either constant or was changing every 50 ms according to different statistical distributions. Adaptation was measured for 28 listeners (9 men) and was quantified as the recognition improvement in the precursor relative to the no-precursor condition. Adaptation was largest for constant-level precursors and did not occur for highly fluctuating precursors, even when the two types of precursors had the same mean level and both activated the medial olivocochlear reflex. Instantaneous amplitude compression of the highly fluctuating precursor produced as much adaptation as the constant-level precursor did without compression. Together, results suggest that noise adaptation in speech recognition is probably mediated by neural dynamic range adaptation to the most frequent sound level. Further, they suggest that auditory peripheral compression per se, rather than the medial olivocochlear reflex, could facilitate noise adaptation by reducing the level fluctuations in the noise.SIGNIFICANCE STATEMENT Recognizing speech in noise is challenging but can be facilitated by noise adaptation. The neural mechanisms underlying this adaptation remain unclear. Here, we report some benefits of adaptation for word-in-noise recognition and show that (1) adaptation occurs for stationary but not for highly fluctuating precursors with equal mean level; (2) both stationary and highly fluctuating noises activate the medial olivocochlear reflex; and (3) adaptation occurs even for highly fluctuating precursors when the stimuli are passed through a fast amplitude compressor. These findings suggest that noise adaptation reflects neural dynamic range adaptation to the most frequent noise level and that auditory peripheral compression, rather than the medial olivocochlear reflex, could facilitate noise adaptation.

Chinese question speech recognition integrated with domain characteristics

Aiming at domain adaptation in speech recognition, we propose a speech recognition method for Chinese question sentence based on domain characteristics. Firstly, by virtue of syllable association characteristics implied in domain term, syllable feature sequences of domain terms are used to construct the domain acoustic model. Secondly, in decoding process of domain-specific Chinese question speech recognition, we utilise domain knowledge relationship to optimise and prune the speech decoding network generated by language model, to improve continuous speech recognition. The experiments on tourist domain corpus show that the proposed method gets the accuracy of 80.50% on Chinese question speech recognition and the accuracy of 91.50% on domain term recognition.

Eigennoise Speech Recovery in Adverse Environments with Joint Compensation of Additive and Convolutive Noise

The learning-based speech recovery approach using statistical spectral conversion has been used for some kind of distorted speech as alaryngeal speech and body-conducted speech (or bone-conducted speech). This approach attempts to recover clean speech (undistorted speech) from noisy speech (distorted speech) by converting the statistical models of noisy speech into that of clean speech without the prior knowledge on characteristics and distributions of noise source. Presently, this approach has still not attracted many researchers to apply in general noisy speech enhancement because of some major problems: those are the difficulties of noise adaptation and the lack of noise robust synthesizable features in different noisy environments. In this paper, we adopted the methods of state-of-the-art voice conversions and speaker adaptation in speech recognition to the proposed speech recovery approach applied in different kinds of noisy environment, especially in adverse environments with joint compensation of additive and convolutive noises. We proposed to use the decorrelated wavelet packet coefficients as a low-dimensional robust synthesizable feature under noisy environments. We also proposed a noise adaptation for speech recovery with the eigennoise similar to the eigenvoice in voice conversion. The experimental results showed that the proposed approach highly outperformed traditional nonlearning-based approaches.

Fast Adaptation of Deep Neural Network Based on Discriminant Codes for Speech Recognition

Fast adaptation of deep neural networks (DNN) is an important research topic in deep learning. In this paper, we have proposed a general adaptation scheme for DNN based on discriminant condition codes, which are directly fed to various layers of a pre-trained DNN through a new set of connection weights. Moreover, we present several training methods to learn connection weights from training data as well as the corresponding adaptation methods to learn new condition code from adaptation data for each new test condition. In this work, the fast adaptation scheme is applied to supervised speaker adaptation in speech recognition based on either frame-level cross-entropy or sequence-level maximum mutual information training criterion. We have proposed three different ways to apply this adaptation scheme based on the so-called speaker codes: i) Nonlinear feature normalization in feature space; ii) Direct model adaptation of DNN based on speaker codes; iii) Joint speaker adaptive training with speaker codes. We have evaluated the proposed adaptation methods in two standard speech recognition tasks, namely TIMIT phone recognition and large vocabulary speech recognition in the Switchboard task. Experimental results have shown that all three methods are quite effective to adapt large DNN models using only a small amount of adaptation data. For example, the Switchboard results have shown that the proposed speaker-code-based adaptation methods may achieve up to 8-10% relative error reduction using only a few dozens of adaptation utterances per speaker. Finally, we have achieved very good performance in Switchboard (12.1% in WER) after speaker adaptation using sequence training criterion, which is very close to the best performance reported in this task (“Deep convolutional neural networks for LVCSR,” T. N. Sainath , Proc. IEEE Acoust., Speech, Signal Process., 2013).

Acoustic Model Adaptation Based on Tensor Analysis of Training Models

We present a tensor analysis of acoustic models comprising various speakers in multiple noise conditions, and its application to the new speaker and environment adaptation for speech recognition. The bases used in adaptation are constructed by decomposing the training models in the state, feature dimension, speaker, and noise spaces using multilinear singular value decomposition. The isolated-word recognition experiment demonstrated the effectiveness of the proposed method, showing better performance than eigenvoice in the babble and factory floor noises for the adaptation data longer than approximately 20 s.

Topic tracking language model for speech recognition

In a real environment, acoustic and language features often vary depending on the speakers, speaking styles and topic changes. To accommodate these changes, speech recognition approaches that include the incremental tracking of changing environments have attracted attention. This paper proposes a topic tracking language model that can adaptively track changes in topics based on current text information and previously estimated topic models in an on-line manner. The proposed model is applied to language model adaptation in speech recognition. We use the MIT OpenCourseWare corpus and Corpus of Spontaneous Japanese in speech recognition experiments, and show the effectiveness of the proposed method.

Systems and methods for providing online fast speaker adaptation in speech recognition

A system ( 230 ) performs speaker adaptation when performing speech recognition. The system ( 230 ) receives an audio segment and identifies the audio segment as a first audio segment or a subsequent audio segment associated with a speaker turn. The system ( 230 ) then decodes the audio segment to generate a transcription associated with the first audio segment when the audio segment is the first audio segment and estimates a transformation matrix based on the transcription associated with the first audio segment. The system ( 230 ) decodes the audio segment using the transformation matrix to generate a transcription associated with the subsequent audio segment when the audio segment is the subsequent audio segment.

Environment adaptation for speech recognition in a speech communication system

A speech communication system comprising a speech input terminal and a speech recognition apparatus which can communicate with each other through a wire or wireless communication network wherein the speech input terminal comprises speech input unit, a unit for creating environment information for speech recognition, which is unique to the speech input terminal or represents its operation state, and a communication control unit for transmitting the environment information to the speech recognition apparatus, and the speech recognition apparatus executes speech recognition processing on the basis of the environment information.

Systematic speaker variation and within-speaker center of gravity correlations in the TIMIT database

The systematicity of speaker variation in consonants was examined by measuring the noise component of consonants in the TIMIT database. Fricatives and stops were compared by measuring at the temporal middle of fricatives and at the release of stops. There is a high correlation between many of the center of gravity measures. For example, if a speaker has a particularly high center of gravity in the sound /t/, she or he will also have a high center of gravity in /d/, /s/, /p/, /k/, and other consonants. The full set of correlations will be described in the paper. The correlations appear to stem from individual differences and not from dialect variations. The implications of the results for rapid speaker adaptation in speech recognition will be explored. [Work supported by San Jose State University Faculty Grant.]

Speech recognition system using neural networks

A speech recognition system can recognize a plurality of voice data having different patterns. The speech recognition system has a voice recognizing and processing device including a plurality of speech recognition neural networks that have previously learned different voice patterns to recognize given voice data. Each of the speech recognition neutral networks is adapted to judge whether or not input voice data coincides with one of the voice data to be recognized. Each neural network then outputs adaptation judgment data representing the adaptation in speech recognition. A selector responsive to the adaptation judgment data from each of the speech recognition neural networks selects one of the neural networks that has the highest adaptation in speech recognition. An output control device outputs the result of speech recognition from the speech recognition neural network selected by the selector.

Future directions in speech information processing

Speech processing technologies, including speech recognition, synthesis, and coding are expected to play important roles in an advanced multimedia society with user-friendly human–machine interfaces. Speech recognition systems include not only those that recognize messages but also those that recognize the identity of the speaker. This paper predicts future directions in speech processing. It describes the most important research problems and tries to forecast where progress will be made in the near future and what applications will become commonplace as a result of the increased capabilities. The most promising application area is telecommunications. To solve various fundamental problems, a unified approach across speech recognition, synthesis, and coding is indispensable. Handling the common phenomenon of voice individuality from different aspects are: research on speaker adaptation in speech recognition, automatic speaker verification, voice conversion in speech synthesis, and the problems of very low-bit-rate speech coding quality variation from speaker to speaker. The methods that essentially solve such problems should be based on a common mathematical model. Research on the mechanism of speech information processing in our human brains, that is, how meanings of speech are understood and how speech is produced, is also crucial for epoch-making technological development in conversational speech understanding and natural speech synthesis.

Speaker adaptation in speech recognition using linear regression techniques

A new technique of speaker adaptation for use in speaker-independent speech recognition systems is presented. The training data is used to build models (based on linear regression) of sounds. At recognition time, the models are used together with an incomplete set of sounds from a new speaker to estimate values for unheard sounds, which are used to adapt the speaker-independent models.

Speaker-dependent-feature extraction, recognition and processing techniques

This paper discusses recent advances in and perspectives of research on speaker-dependent-feature extraction from speech waves, automatic speaker identification and verification, speaker adaptation in speech recognition, and voice conversion techniques. Speaker-dependent information exists both in the spectral envelope and in the supra-segmental features of speech. This individual information can be further classified into temporal and dynamic features. Speaker identification/verification methods can be divided into text-dependent and tect-independent methods. Although text-dependent speaker verification techniques have almost reached the level suitable for practical implementation, text-independent techniques are still in the fundamental research stage. Both supervised and unsupervised speaker adaptation algorithms for speech recognition have recently been proposed, and remarkable progress has been achieved in this field. Improving synthesized speech quality by adding natural characteristics of voice individuality, and converting synthesized voice individuality from one speaker to another, are as yet little exploited research fields to be studied in the near future. Research on speaker-dependent information is one of the most important future directions for achieving advanced speech information processing systems.

Pronunciation variation within and across speakers

An understanding of the structure of pronunciation variation over a population of speakers and over time in the utterances of one speaker should be useful in designing speaker-independent speech recognizers. This paper reports a series of experiments designed to show different kinds of patterns observed in alternative forms of words in constant contexts (e.g., the presence or absence of frication in the “y” in “had your”). An analysis is presented of transcribed data from 630 speakers reading two sample sentences as well as data from four speakers reading the same two sentences 24 times each, separated by filler material, in three separate sessions. The analysis quantifies the relative usefulness of competing models of variation in information theoretic terms. The results indicate that (1) speakers can be clustered into low variation groups such that the variation within a group is significantly less than the population variation, and (2) individual speakers show greater consistency than comparable clustered subsets of the population. Finally, it is suggested how this structure may be used to guide rapid, automatic adaptation in speech recognition. [Work supported by NSE.]