Telephone Speech Research Articles

Discriminative cluster adaptive training

Multiple-cluster schemes, such as cluster adaptive training (CAT) or eigenvoice systems, are a popular approach for rapid speaker and environment adaptation. Interpolation weights are used to transform a multiple-cluster, canonical, model to a standard hidden Markov model (HMM) set representative of an individual speaker or acoustic environment. Maximum likelihood training for CAT has previously been investigated. However, in state-of-the-art large vocabulary continuous speech recognition systems, discriminative training is commonly employed. This paper investigates applying discriminative training to multiple-cluster systems. In particular, minimum phone error (MPE) update formulae for CAT systems are derived. In order to use MPE in this case, modifications to the standard MPE smoothing function and the prior distribution associated with MPE training are required. A more complex adaptive training scheme combining both interpolation weights and linear transforms, a structured transform (ST), is also discussed within the MPE training framework. Discriminatively trained CAT and ST systems were evaluated on a state-of-the-art conversational telephone speech task. These multiple-cluster systems were found to outperform both standard and adaptively trained systems

Analysis of the errors produced by the 2004 BBN speech recognition system in the DARPA EARS evaluations

This paper aims to quantify the main error types the 2004 BBN speech recognition system made in the broadcast news (BN) and conversational telephone speech (CTS) DARPA EARS evaluations. We show that many of the remaining errors occur in clusters rather than isolated, have specific causes, and differ to some extent between the BN and CTS domains. The correctly recognized words are also clustered and are highly correlated with regions where the system produces a single hypothesized choice per word. A statistical analysis of some well-known error causes (out-of-vocabulary words, word fragments, hesitations, and unlikely language constructs) was performed in order to assess their contribution to the overall word error rate (WER). We conclude with a discussion of the lower bound on the WER introduced by the human annotator disagreement

Opinions on Cochlear Implant Use in Senior MED-EL Patients

This study assessed the subjective benefits of cochlear implants in senior patients wearing a MED-EL device. Data was compared with previous studies to assess the influence of recent speech-coding strategies and behind-the-ear speech processors; users of the behind-the-ear device and the body-worn device were also compared. An adaptation of the Nucleus 22-channel survey was sent to 141 cochlear implant users. The survey assesses perception of the device, communication benefits, handling the device and quality of life. Ninety-five surveys were returned. Results demonstrate that advanced technology provides greater benefit now than 9–13 years ago, notably: improved listening across noise, better understanding on the telephone and better speech perception. Users of the behind-the-ear device did not report more difficulties than body-worn device users but demonstrated better performance. Results show a positive outcome for cochlear implantation in a MED-EL seniors group.

Comparison of different mobile telephones in Combi40 + users

Conclusion. Telephone usage is a difficult challenge for cochlear implanted patients. A significant number of cochlear implanted patients are able to carry on a land-line or mobile telephone conversation. The telephone model may be a critical factor for telephone performance. Objectives. The goals of this study were to evaluate speech discrimination through land-line and mobile telephones in Combi40 + users, and to compare different mobile telephone models to find out which could be more advisable for them. Patients and methods. Eighteen Combi40 + implantees were tested with CID sentences and bisyllabic words presented through land-line and mobile telephones, in both quiet and noisy environments. Results. Mean scores for telephonic speech discrimination were over 85% for CID sentences and 28–59% for bisyllabic words. The Siemens M55 was superior to the other mobile telephones tested.

On variable-scale piecewise stationary spectral analysis of speech signals for ASR

It is often acknowledged that speech signals contain short-term and long-term temporal properties [Rabiner, L., Juang, B.H., 1993. Fundamentals of Speech Recognition, Prentice-Hall, NJ, USA] that are difficult to capture and model by using the usual fixed scale (typically 20 ms) short-time spectral analysis used in hidden Markov models (HMMs), based on piecewise stationarity and state conditional independence assumptions of acoustic vectors. For example, vowels are typically quasi-stationary over 40–80 ms segments, while plosive typically require analysis below 20 ms segments. Thus, fixed scale analysis is clearly sub-optimal for “optimal” time–frequency resolution and modeling of different stationary phones found in the speech signal. In the present paper, we investigate the potential advantages of using variable size analysis windows towards improving state-of-the-art speech recognition systems. Based on the usual assumption that the speech signal can be modeled by a time-varying autoregressive (AR) Gaussian process, we estimate the largest piecewise quasi-stationary speech segments, based on the likelihood that a segment was generated by the same AR process. This likelihood is estimated from the linear prediction (LP) residual error. Each of these quasi-stationary segments is then used as an analysis window from which spectral features are extracted. Such an approach thus results in a variable-scale time spectral analysis, adaptively estimating the largest possible analysis window size such that the signal remains quasi-stationary, thus the best temporal/frequency resolution tradeoff. The speech recognition experiments on the OGI Numbers95 database [Cole, R.A., Fanty, M., Lander, T., 1994. Telephone speech corpus at CSLU. In: Proc. of ICSLP, Yokohama, Japan.], show that the proposed variable-scale piecewise stationary spectral analysis based features indeed yield improved recognition accuracy in clean conditions, compared to features based on minimum cross entropy spectrum [Loughlin, P., Pitton, J., Hannaford, B., 1994. Approximating time–frequency density functions via optimal combinations of spectrograms, IEEE Signal Process. Lett. 1 (12)] as well as those based on fixed scale spectral analysis.

Design, implementation and evaluation of the Czech realistic audio-visual speech synthesis

This paper presents the whole process of creation of audio-visual speech synthesis system. Such system consists of two main parts, the acoustic synthesis emulating human speech and the facial animation emulating the human lip articulation. The acoustic subsystem is based on concatenation-based speech synthesis. The visual subsystem is designed as a realistic, fully three-dimensional parametrically controllable facial animation model. To be able to parametrically control the animation to emulate human articulation, the set of visual parameters has to be obtained for all basic speech units. To provide realistic animation, the database of lip movements of a real person need to be recorded and expressed by suitable parameterization. The set of control parameters for visual animation is then derived from this database. The 3D model of a head based on a head of a real person also makes the animation more realistic. To obtain such model, a 3D scanning of a real person has to be adopted. We present the design and implementation of above-mentioned process. The aim is to obtain realistic audio-visual speech synthesis with possibility to change the 3D head model according to particular person. The design, acquisition and processing of audio-visual speech corpus for such purpose is presented. Next, the process of both acoustic and visual speech synthesis is described. The visual speech synthesis comprises the tasks of model training, animation control, and co-articulation modelling. A facial animation can also increase intelligibility of a telephone speech to people with hearing disabilities. In such case the textual information to control the animation is not available. Solution to the problem of mapping visual parameters from speech signal either directly or through recognized text is presented. Furthermore, the 3D scanning algorithm is presented. It allows to obtain realistic 3D model based on a head of a real person and thus to personalize the talking head. In the end of this paper, evaluation of intelligibility of the presented audio-visual speech synthesis and its possible applications are presented.

Noise robust bandwidth extension of telephone speech for mobile and landline communications

The present day telephone infrastructure limits the bandwidth of telephone speech to 300–3400 Hz, whereby conversational speech falls mostly within 50–8000 Hz. The ensuing band-limited narrow-band (NB) speech has a considerable loss in sound quality, in terms of naturalness and intelligibility. The purpose of bandwidth extension (BWE) is to utilize the receiver end NB speech and to add relevant frequency information to the missing low and high frequencies. This novel BWE algorithm utilizes the linear source filter model, which separates NB speech into two roughly independent components, the excitation and the spectral envelope. These components are then extended and convolved together to attain reconstructed wideband (RWB) speech. Excitation extension involves harmonic reconstruction for voiced frames and some modulations and filtering steps for unvoiced frames. Spectral envelope extension utilizes a Gaussian mixture model trained via the expectation maximization algorithm, which is involved in the mapping of NB noise robust features to the corresponding RWB features. Noise analysis is used to isolate the best feature set for real world applications. Results from a subjective listening test indicate this algorithm has an improvement in sound quality, while having the lowest known distortion metric among other speaker independent bandwidth extension algorithms.

Minimum phone error training of precision matrix models

Gaussian mixture models (GMMs) are commonly used as the output density function for large-vocabulary continuous speech recognition (LVCSR) systems. A standard problem when using multivariate GMMs to classify data is how to accurately represent the correlations in the feature vector. Full covariance matrices yield a good model, but dramatically increase the number of model parameters. Hence, diagonal covariance matrices are commonly used. Structured precision matrix approximations provide an alternative, flexible, and compact representation. Schemes in this category include the extended maximum likelihood linear transform and subspace for precision and mean models. This paper examines how these precision matrix models can be discriminatively trained and used on state-of-the-art speech recognition tasks. In particular, the use of the minimum phone error criterion is investigated. Implementation issues associated with building LVCSR systems are also addressed. These models are evaluated and compared using large vocabulary continuous telephone speech and broadcast news English tasks.

Interruption/Interaction/Collaboration: A Critical Appraisal of the Textual @traction Interactive Event

Interruption/Interaction/Collaboration: A Critical Appraisal of the Textual @traction Interactive Event

Corrections to “Automatic Transcription of Conversational Telephone Speech”

Corrections to “Automatic Transcription of Conversational Telephone Speech”

Random forests and the data sparseness problem in language modeling

Language modeling is the problem of predicting words based on histories containing words already hypothesized. Two key aspects of language modeling are effective history equivalence classification and robust probability estimation. The solution of these aspects is hindered by the data sparseness problem. Application of random forests (RFs) to language modeling deals with the two aspects simultaneously. We develop a new smoothing technique based on randomly grown decision trees (DTs) and apply the resulting RF language models to automatic speech recognition. This new method is complementary to many existing ones dealing with the data sparseness problem. We study our RF approach in the context of n-gram type language modeling in which n − 1 words are present in a history. Unlike regular n-gram language models, RF language models have the potential to generalize well to unseen data, even when histories are longer than four words. We show that our RF language models are superior to the best known smoothing technique, the interpolated Kneser–Ney smoothing, in reducing both the perplexity (PPL) and word error rate (WER) in large vocabulary state-of-the-art speech recognition systems. In particular, we will show statistically significant improvements in a contemporary conversational telephony speech recognition system by applying the RF approach only to one of its many language models.

Language identification using acoustic log-likelihoods of syllable-like units

Automatic spoken language identification (LID) is the task of identifying the language from a short utterance of the speech signal uttered by an unknown speaker. The most successful approach to LID uses phone recognizers of several languages in parallel [Zissman, M.A., 1996. Comparison of four approaches to automatic language identification of telephone speech. IEEE Trans. Speech Audio Process. 4 (1), 31–44]. The basic requirement to build a parallel phone recognition (PPR) system is segmented and labeled speech corpora. In this paper, a novel approach is proposed for the LID task which uses parallel syllable-like unit recognizers, in a frame work similar to the PPR approach in the literature. The difference is that the sub-word unit models for each of the languages to be recognized are generated in an unsupervised manner without the use of segmented and labeled speech corpora. The training data of each of the languages is first segmented into syllable-like units and language-dependent syllable-like unit inventory is created. These syllable-like units are then clustered using an incremental approach. This results in a set of syllable-like units models for each language. Using these language-dependent syllable-like unit models, language identification is performed based on accumulated acoustic log-likelihoods. Our initial results on the Oregon Graduate Institute Multi-language Telephone Speech Corpus [Muthusamy, Y.K., Cole, R.A., Oshika, B.T., 1992. The OGI multi-language telephone speech corpus. In: Proceedings of Internat. Conf. Spoken Language Process., October 1992, pp. 895–898] show that the performance is 72.3%. We further show that if only a subset of syllable-like unit models that are unique (in some sense) are considered, the performance improves to 75.9%.

Automatic transcription of conversational telephone speech

This paper discusses the Cambridge University HTK (CU-HTK) system for the automatic transcription of conversational telephone speech. A detailed discussion of the most important techniques in front-end processing, acoustic modeling and model training, language and pronunciation modeling are presented. These include the use of conversation side based cepstral normalization, vocal tract length normalization, heteroscedastic linear discriminant analysis for feature projection, minimum phone error training and speaker adaptive training, lattice-based model adaptation, confusion network based decoding and confidence score estimation, pronunciation selection, language model interpolation, and class based language models. The transcription system developed for participation in the 2002 NIST Rich Transcription evaluations of English conversational telephone speech data is presented in detail. In this evaluation the CU-HTK system gave an overall word error rate of 23.9%, which was the best performance by a statistically significant margin. Further details on the derivation of faster systems with moderate performance degradation are discussed in the context of the 2002 CU-HTK 10 /spl times/ RT conversational speech transcription system.

Techniques for artificial bandwidth extension of telephone speech

Most of the current telephone systems are still restricted to the historically motivated limitation of the bandwidth from 0.3 to 3.4 kHz. This bandwidth ensures a sufficient intelligibility of speech—the speech quality, however, suffers from the reduced bandwidth. One possible approach to reduce the quality loss is the application of an artificial bandwidth extension system at the receiver which is proposed in this contribution. Based on a priori knowledge concerning the characteristics of speech signals and without transmitting any extra information the bandwidth is extended to the range from the lower hearing limit to 8 kHz. Because of the uniqueness of speech an exact reconstruction of the spectrally unrestricted original signal is not possible. In fact, only the subjective impression of an enlarged bandwidth is possible and intended. In this contribution a new method for artificial bandwidth extension of telephone speech towards frequencies below and above the telephone-band is proposed. It is based on a simple codebook approach which on its own does not ensure a satisfying quality of the processed signal. The proposed new method extends this approach by several optional components to reduce the artifacts. These additional components comprise time-smoothing, group-averaging, a more sophisticated classification approach including memory and a separate technique for low-pass extension.

Unsupervised speaker indexing using generic models

Unsupervised speaker indexing sequentially detects points where a speaker identity changes in a multispeaker audio stream, and categorizes each speaker segment, without any prior knowledge about the speakers. This paper addresses two challenges: The first relates to sequential speaker change detection. The second relates to speaker modeling in light of the fact that the number/identity of the speakers is unknown. To address this issue, a predetermined generic speaker-independent model set, called the sample speaker models (SSM), is proposed. This set can be useful for more accurate speaker modeling and clustering without requiring training models on target speaker data. Once a speaker-independent model is selected from the generic sample models, it is progressively adapted into a specific speaker-dependent model. Experiments were performed with data from the Speaker Recognition Benchmark NIST Speech corpus (1999) and the HUB-4 Broadcast News Evaluation English Test material (1999). Results showed that our new technique, sampled using the Markov Chain Monte Carlo method, gave 92.5% indexing accuracy on two speaker telephone conversations, 89.6% on four-speaker conversations with the telephone speech quality, and 87.2% on broadcast news. The SSMs outperformed the universal background model by up to 29.4% and the universal gender models by up to 22.5% in indexing accuracy in the experiments of this paper.

Aural and automatic forensic speaker recognition in mismatched conditions

In this article, we compare aural and automatic speaker recognition in the context of forensic analyses, using a Bayesian framework for the interpretation of evidence.We use perceptual tests performed by non-experts and compare their performance with that of an automatic speaker recognition system. These experiments are performed with 90 phonetically untrained subjects. Several forensic cases were simulated, using the Polyphone IPSC-02 database, varying in linguistic content and technical conditions of recording. We estimate the strength of evidence for both humans and the baseline automatic system, calculating likelihood ratios using perceptual scores for humans and log-likelihood scores for the automatic system. A methodology analogous to the Bayesian interpretation in forensic automatic speaker recognition is applied to the perceptual scores given by humans in order to estimate the strength of evidence. The degradation of the accuracy of human recognition in mismatched recording conditions is contrasted with that of the automatic system under similar recording conditions. The conditions considered are fixed telephone, cellular telephone and noisy speech in forensically realistic conditions. The perceptual cues that the human subjects use to perceive differences in voices are studied, along with their importance in different recording conditions. We observe that while automatic speaker recognition shows higher accuracy in matched conditions of training and testing, its performance degrades significantly in mismatched conditions. Aural recognition accuracy is also observed to degrade from matched conditions to mismatched conditions and in mismatched conditions, the baseline automatic systems showed comparable or slightly degraded performance compared to the aural recognition systems. The baseline automatic system with adaptation to noisy conditions showed comparable or better performance than aural recognition. The higher level perceptual cues used by human listeners in order to recognise speakers are discussed. We also discuss the possibility of increasing the accuracy of automatic systems using the perceptual cues that remain robust to mismatched recording conditions.

A study in machine learning from imbalanced data for sentence boundary detection in speech

Enriching speech recognition output with sentence boundaries improves its human readability and enables further processing by downstream language processing modules. We have constructed a hidden Markov model (HMM) system to detect sentence boundaries that uses both prosodic and textual information. Since there are more nonsentence boundaries than sentence boundaries in the data, the prosody model, which is implemented as a decision tree classifier, must be constructed to effectively learn from the imbalanced data distribution. To address this problem, we investigate a variety of sampling approaches and a bagging scheme. A pilot study was carried out to select methods to apply to the full NIST sentence boundary evaluation task across two corpora (conversational telephone speech and broadcast news speech), using both human transcriptions and recognition output. In the pilot study, when classification error rate is the performance measure, using the original training set achieves the best performance among the sampling methods, and an ensemble of multiple classifiers from different downsampled training sets achieves slightly poorer performance, but has the potential to reduce computational effort. However, when performance is measured using receiver operating characteristics (ROC) or area under the curve (AUC), then the sampling approaches outperform the original training set. This observation is important if the sentence boundary detection output is used by downstream language processing modules. Bagging was found to significantly improve system performance for each of the sampling methods. The gain from these methods may be diminished when the prosody model is combined with the language model, which is a strong knowledge source for the sentence detection task. The patterns found in the pilot study were replicated in the full NIST evaluation task. The conclusions may be dependent on the task, the classifiers, and the knowledge combination approach.

Acoustic correlates of non-modal phonation in telephone speech

Non-modal phonation conveys both linguistic and paralinguistic information, and is distinguished by acoustic source and filter features. Detecting non-modal phonation in speech requires reliable F0 analysis, a problem for telephone-band speech, where F0 analysis frequently fails. An approach is demonstrated to the detection of creaky phonation in telephone speech based on robust F0 and spectral analysis. The F0 analysis relies on an autocorrelation algorithm applied to the inverse-filtered speech signal and succeeds in regions of non-modal phonation where the non-filtered F0 analysis typically fails. In addition to the extracted F0 values, spectral amplitude is measured at the first two harmonics (H1, H2) and the first three formants (A1, A2, A3). F0 and spectral tilt are measured from 300 samples of modal and creaky voice vowels, selected from Switchboard telephone speech using auditory and visual criteria. Results show successful F0 detection in creaky voice regions, with distinctive low F0, and statistically significant differences between modal and creaky voice in measures of spectral amplitude, especially for measures based on H1. Our current work develops methods for the automatic detection of creaky voicing in spontaneous speech based on the analysis technique shown here. [Work supported by NSF.]

A novel approach for modeling non-keyword intervals in a keyword spotter exploiting acoustic similarities of languages

In this paper, we present a new keyword spotting technique. A critical issue in keyword spotting is the explicit modeling of the non-keyword portions. To date, most keyword spotters use a set of Hidden Markov Models (HMM) to represent the non-keyword portions. A widely used approach is to split the training data into keyword and non-keyword data. The keywords are represented by HMMs trained using the keyword speech, and the garbage models are trained using the non-keyword speech. The main disadvantage of this method is the task dependence. Another approach is to use a common set of acoustic models for both keywords and garbage models. However, this method faces a major problem. In a keyword spotter, the garbage models are usually connected to allow any sequence. Therefore, the keywords are also included in these sequences. When the same training data are used for keyword and garbage models, the garbage models also cover the keywords. In order to overcome these problems, we propose a new method for modeling the non-keyword intervals. In our method, the garbage models are phonemic HMMs trained using a speech corpus of a language other than—but acoustically similar to—the target language. In our work, the target language is Japanese and, due to the high similarity, English was chosen as the ‘garbage language’ for training the garbage models. Using English garbage models—instead of Japanese—our method achieves higher performance, compared with when Japanese garbage models are used. Moreover, parameter tuning (e.g., word insertion penalty) does not have a serious effect on the performance when English garbage models are used. Using clean telephone speech test data and a vocabulary of 100 keywords, we achieved a 7.9% equal error rate which is a very promising result. In this paper we also introduce results obtained using several sizes of vocabulary, and we investigate the selection of the most appropriate garbage model set. In addition to the Japanese keyword spotting system, we also introduce results of an English keyword spotter. By using Japanese garbage models—instead of English—we achieved significant improvement. Using telephone speech test data and a vocabulary of 25 keywords the achieved Figure of Merit (FOM) was 74.7% compared to 68.9% when English garbage models were used.

Discriminación verbal a través del teléfono en pacientes implantados con un Combi 40+

ObjectiveTo assess speech discrimination through line and mobile telephone in cochlear implanted patients. Material and MethodsEighteen patients implanted with a Combi 40+ went through different speech discrimination tests with a line telephone and three mobile telephones, in quiet and noisy environment. ResultsMean scores for telephonic speech discrimination were 84-92% using CID sentences, 27-58% using bisyllabic words in quiet environment and 15-41% using bisyllabic words in noisy environment. Among the mobile telephones tested, the Siemens m55 reached the best scores. ConclusionsTelephonic speech discrimination is achieved by a significant number of cochlear implanted patients. Certain mobile telephone models appear to be more advisable for these patients.