Telephone Speech Research Articles

Spoken Term Detection Based on Unconstrained Word Graph Extension

The performance of keyword spotting system suffers severe degradation when the index stage is so fast that the lattice may lose lots of information to retrieve the spoken terms. In this paper, we focus on this problem and present an algorithm called Unconstraint Word Graph Expansion (UWGE) to keep the pruned hypotheses which are discarded in the decoding procedure but may contain correct hypotheses. The proposed approach is to eliminate the N-gram language model state limitation of lattice and reconstruct lattice to unconstrained word graph. On two Mandarin conversation telephone speech sets, we compare performance using UWGE with that on traditional trigram lattice, and our approach gives satisfying performance gains over trigram lattice. We also show the relationship between the performance and the system speed based on this approach.

Assessment of telephone speech perception in individuals who received cochlear implant in the period 1993-2003

To evaluate telephone speech perception in individuals who received cochlear implant in the period 1993-2003. Twenty seven CI users were divided into pre and post-lingual groups, being the speech perception assessed in two stages: first by a list of sentences imposed on speakerphone with the same mapping used to evaluate the phone and, in a second stage, using the landline, landline phone adapter with CI and cell phone. In the group of pre-lingual hearing loss, 75% of subjects were able to maintain a dialogue with the interlocutor and 19% did so with difficulty. In the post-lingual group, 89% were able to maintain the dialogue with the interlocutor and 11% did so with difficulty. Both groups of subjects pre and post-lingual use the phone as a media, and most have satisfactory performance without the need for aid or CI accessories. One of the benefits of the CI is to introduce the life of the hearing impaired phone use regardless of their technology and accessories, as well as with the phone adapter and cell phone. In this study, most individuals CI users showed satisfactory performance in the comprehension of sentence and questions by the telephone, with better performance, although discrete, for the group post-lingual. Thus, there is the importance of careful indication of the IC and auditory rehabilitation program that minimize the impact of deafness in the communication, especially via telephone, regardless of the time when the hearing was acquired. So, this fact improves the conditions for CI users to maintain their daily activities, following the dynamism of contemporary life.

The Perception of Telephone-Processed Speech by Combined Electric and Acoustic Stimulation

This study assesses the effects of adding low- or high-frequency information to the band-limited telephone-processed speech on bimodal listeners' telephone speech perception in quiet environments. In the proposed experiments, bimodal users were presented under quiet listening conditions with wideband speech (WB), bandpass-filtered telephone speech (300-3,400 Hz, BP), high-pass filtered speech (f > 300 Hz, HP, i.e., distorted frequency components above 3,400 Hz in telephone speech were restored), and low-pass filtered speech (f < 3,400 Hz, LP, i.e., distorted frequency components below 300 Hz in telephone speech were restored). Results indicated that in quiet environments, for all four types of stimuli, listening with both hearing aid (HA) and cochlear implant (CI) was significantly better than listening with CI alone. For both bimodal and CI-alone modes, there were no statistically significant differences between the LP and BP scores and between the WB and HP scores. However, the HP scores were significantly better than the BP scores. In quiet conditions, both CI alone and bimodal listening achieved the largest benefits when telephone speech was augmented with high rather than low-frequency information. These findings provide support for the design of algorithms that would extend higher frequency information, at least in quiet environments.

Long Mandarin Spoken Term Detection Using Two-Stage Search

For efficient collection of speech recordings, the ability to search for spoken terms in the speech stream is an essential capability. Although the Chinese spoken term detection (STD) does not suffer the out-of-vocabulary (OOV) problem as English, it is still hard to retrieve the long spoken terms which contain four characters or more. In this paper, we details our approach for long Mandarin spoken term detection which combines the search on inverted index produced by speech recognizer and linear scan on syllable confusion network. First, we split the long spoken terms into syllables and search the syllables on the inverted index _le to get the segments which may contain the long spoken terms. Then we use a linear scan algorithm on syllable confusion networks (SCNs). On two Mandarin conversation telephone speech sets, we compare performance using the method proposed with that of the baseline syllable-based systems, and our approach gives satisfying performance gains over the others.

A study of voice activity detection techniques for NIST speaker recognition evaluations

Since 2008, interview-style speech has become an important part of the NIST speaker recognition evaluations (SREs). Unlike telephone speech, interview speech has lower signal-to-noise ratio, which necessitates robust voice activity detectors (VADs). This paper highlights the characteristics of interview speech files in NIST SREs and discusses the difficulties in performing speech/non-speech segmentation in these files. To overcome these difficulties, this paper proposes using speech enhancement techniques as a pre-processing step for enhancing the reliability of energy-based and statistical-model-based VADs. A decision strategy is also proposed to overcome the undesirable effects caused by impulsive signals and sinusoidal background signals. The proposed VAD is compared with the ASR transcripts provided by NIST, VAD in the ETSI-AMR Option 2 coder, satistical-model (SM) based VAD, and Gaussian mixture model (GMM) based VAD. Experimental results based on the NIST 2010 SRE dataset suggest that the proposed VAD outperforms these conventional ones whenever interview-style speech is involved. This study also demonstrates that (1) noise reduction is vital for energy-based VAD under low SNR; (2) the ASR transcripts and ETSI-AMR speech coder do not produce accurate speech and non-speech segmentations; and (3) spectral subtraction makes better use of background spectra than the likelihood-ratio tests in the SM-based VAD. The segmentation files produced by the proposed VAD can be found in http://bioinfo.eie.polyu.edu.hk/ssvad.

Speech perception and communication ability over the telephone by Mandarin-speaking children with cochlear implants

Objective(1) To understand speech perception and communication ability through real telephone calls by Mandarin-speaking children with cochlear implants and compare them to live-voice perception, (2) to report the general condition of telephone use of this population, and (3) to investigate the factors that correlate with telephone speech perception performance. MethodsFifty-six children with over 4 years of implant use (aged 6.8–13.6 years, mean duration 8.0 years) took three speech perception tests administered using telephone and live voice to examine sentence, monosyllabic-word and Mandarin tone perception. The children also filled out a questionnaire survey investigating everyday telephone use. Wilcoxon signed-rank test was used to compare the scores between live-voice and telephone tests, and Pearson's test to examine the correlation between them. ResultsThe mean scores were 86.4%, 69.8% and 70.5% respectively for sentence, word and tone recognition over the telephone. The corresponding live-voice mean scores were 94.3%, 84.0% and 70.8%. Wilcoxon signed-rank test showed the sentence and word scores were significantly different between telephone and live voice test, while the tone recognition scores were not, indicating tone perception was less worsened by telephone transmission than words and sentences. Spearman's test showed that chronological age and duration of implant use were weakly correlated with the perception test scores. The questionnaire survey showed 78% of the children could initiate phone calls and 59% could use the telephone 2 years after implantation. ConclusionImplanted children are potentially capable of using the telephone 2 years after implantation, and communication ability over the telephone becomes satisfactory 4 years after implantation.

An adaptive post-filtering method producing an artificial Lombard-like effect for intelligibility enhancement of narrowband telephone speech

Post-filtering can be used in mobile communications to improve the quality and intelligibility of speech. Energy reallocation with a high-pass type filter has been shown to work effectively in improving the intelligibility of speech in difficult noise conditions. This paper introduces a post-filtering algorithm that adapts to the background noise level as well as to the fundamental frequency of the speaker and models the spectral effects observed in natural Lombard speech. The introduced method and another post-filtering technique were compared to unprocessed telephone speech in subjective listening tests in terms of intelligibility and quality. The results indicate that the proposed method outperforms the reference method in difficult noise conditions.

Spontaneous speech and opinion detection: mining call-centre transcripts

Opinion mining on conversational telephone speech tackles two challenges: the robustness of speech transcriptions and the relevance of opinion models. The two challenges are critical in an industrial context such as marketing. The paper addresses jointly these two issues by analyzing the influence of speech transcription errors on the detection of opinions and business concepts. We present both modules: the speech transcription system, which consists in a successful adaptation of a conversational speech transcription system to call-centre data and the information extraction module, which is based on a semantic modeling of business concepts, opinions and sentiments with complex linguistic rules. Three models of opinions are implemented based on the discourse theory, the appraisal theory and the marketers' expertise, respectively. The influence of speech recognition errors on the information extraction module is evaluated by comparing its outputs on manual versus automatic transcripts. The F-scores obtained are 0.79 for business concepts detection, 0.74 for opinion detection and 0.67 for the extraction of relations between opinions and their target. This result and the in-depth analysis of the errors show the feasibility of opinion detection based on complex rules on call-centre transcripts.

Multi-band summary correlogram-based pitch detection for noisy speech

A multi-band summary correlogram (MBSC)-based pitch detection algorithm (PDA) is proposed. The PDA performs pitch estimation and voiced/unvoiced (V/UV) detection via novel signal processing schemes that are designed to enhance the MBSC’s peaks at the most likely pitch period. These peak-enhancement schemes include comb-filter channel-weighting to yield each individual subband’s summary correlogram (SC) stream, and stream-reliability-weighting to combine these SCs into a single MBSC. V/UV detection is performed by applying a constant threshold on the maximum peak of the enhanced MBSC. Narrowband noisy speech sampled at 8kHz are generated from Keele (development set) and CSTR – Centre for Speech Technology Research-(evaluation set) corpora. Both 4-kHz fullband speech, and G.712-filtered telephone speech are simulated. When evaluated solely on pitch estimation accuracy, assuming voicing detection is perfect, the proposed algorithm has the lowest gross pitch error for noisy speech in the evaluation set among the algorithms evaluated (RAPT, YIN, etc.). The proposed PDA also achieves the lowest average pitch detection error, when both pitch estimation and voicing detection errors are taken into account.

A unified framework for domain independent online speaker indexing in eigen-voice space using an index tree of reference models

Speaker indexing referred in literature as speaker diarization is an important task in audio indexing and retrieval. Speaker indexing includes two important and usually separate stages, namely speaker segmentation and speaker clustering. Speaker indexing can be divided into online and offline categories. This paper mainly focuses on domain independent online speaker indexing. For this purpose, the proposed framework should be parameter free and no application specific parameters such as utterance duration or threshold settings are required. To reduce dependency on parameters, the traditional speaker segmentation is reformed to a voting based homogeneous speech segmentation, in which several approaches are applied in parallel to decide on the existence of a change point. In online indexing, data insufficiency is encountered at each time slice. In the proposed framework, a set of reference speaker models are used as side information to facilitate online tracking. To improve the indexing accuracy, adaptation approaches in eigen-voice decomposition space are proposed in this paper. To enhance the tracking performance from the computational cost point of view, an index structure of the reference models is proposed to speed up the search in the model space. The proposed framework is evaluated on the 2002 Rich Transcription Broadcast News and Conversational Telephone Speech Corpus (in Garofolo, NIST Rich Transcription, 2002) as well as a synthetic dataset. The indexing error of the proposed framework on telephone conversations, broadcast news and synthetic dataset are 7.51 %, 6.36 % and 9.34 %, respectively. Also, using the index tree structure approach, the tracking run time of the proposed framework is improved by 32 %.

Spoken Term Detection Based on Improved Index Structure

The performance of keyword spotting system suffers severe degradation when the index stage is so fast that the lattice may lose lots of information to retrieve the spoken terms. In this paper, we focus on this problem and present two algorithm: the first one called unconstraint word graph expansion (UWGE) and the other called dynamic position specific posterior lattice(D-PSPL). The motivation of these methods is to keep the pruned hypotheses which are discarded in the decoding procedure but may contain correct hypotheses. The proposed approaches is to eliminate the N-gram language model state limitation of lattice and reconstruct lattice to unconstrained word graph. On two Mandarin conversation telephone speech sets, we compare performance using the two methods with that on traditional trigram lattice, and our approaches give satisfying performance gains over trigram lattice. The experiment results also show that the D-PSPL algorithm is better than the UWGE algorithm in high score area.

Discriminative Approach to Build Hybrid Vocabulary for Conversational Telephone Speech Recognition of Agglutinative Languages

Morphemes, which are obtained from morphological parsing, and statistical sub-words, which are derived from data-driven splitting, are commonly used as the recognition units for speech recognition of agglutinative languages. In this letter, we propose a discriminative approach to select the splitting result, which is more likely to improve the recognizer's performance, for each distinct word type. An objective function which involves the unigram language model (LM) probability and the count of misrecognized phones on the acoustic training data is defined and minimized. After determining the splitting result for each word in the text corpus, we select the frequent units to build a hybrid vocabulary including morphemes and statistical sub-words. Compared to a statistical sub-word based system, the hybrid system achieves 0.8% letter error rates (LERs) reduction on the test set.

Signal-to-noise ratio adaptive post-filtering method for intelligibility enhancement of telephone speech

Post-filtering can be utilized to improve the quality and intelligibility of telephone speech. Previous studies have shown that energy reallocation with a high-pass type filter works effectively in improving the intelligibility of speech in difficult noise conditions. The present study introduces a signal-to-noise ratio adaptive post-filtering method that utilizes energy reallocation to transfer energy from the first formant to higher frequencies. The proposed method adapts to the level of the background noise so that, in favorable noise conditions, the post-filter has a flat frequency response and the effect of the post-filtering is increased as the level of the ambient noise increases. The performance of the proposed method is compared with a similar post-filtering algorithm and unprocessed speech in subjective listening tests which evaluate both intelligibility and listener preference. The results indicate that both of the post-filtering methods maintain the quality of speech in negligible noise conditions and are able to provide intelligibility improvement over unprocessed speech in adverse noise conditions. Furthermore, the proposed post-filtering algorithm performs better than the other post-filtering method under evaluation in moderate to difficult noise conditions, where intelligibility improvement is mostly required.

Large-Vocabulary Continuous Speech Recognition Systems: A Look at Some Recent Advances

Over the past decade or so, several advances have been made to the design of modern large vocabulary continuous speech recognition (LVCSR) systems to the point where their application has broadened from early speaker dependent dictation systems to speaker-independent automatic broadcast news transcription and indexing, lectures and meetings transcription, conversational telephone speech transcription, open-domain voice search, medical and legal speech recognition, and call center applications, to name a few. The commercial success of these systems is an impressive testimony to how far research in LVCSR has come, and the aim of this article is to describe some of the technological underpinnings of modern systems. It must be said, however, that, despite the commercial success and widespread adoption, the problem of large-vocabulary speech recognition is far from being solved: background noise, channel distortions, foreign accents, casual and disfluent speech, or unexpected topic change can cause automated systems to make egregious recognition errors. This is because current LVCSR systems are not robust to mismatched training and test conditions and cannot handle context as well as human listeners despite being trained on thousands of hours of speech and billions of words of text.

Bandwidth Extension of Telephone Speech to Low Frequencies Using Sinusoidal Synthesis and a Gaussian Mixture Model

The quality of narrowband telephone speech is degraded by the limited audio bandwidth. This paper describes a method that extends the bandwidth of telephone speech to the frequency range 0-300 Hz. The method generates the lowest harmonics of voiced speech using sinusoidal synthesis. The energy in the extension band is estimated from spectral features using a Gaussian mixture model. The amplitudes and phases of the synthesized sinusoidal components are adjusted based on the amplitudes and phases of the narrowband input speech, which provides adaptivity to varying input bandwidth characteristics. The proposed method was evaluated with listening tests in combination with another bandwidth extension method for the frequency range 4-8 kHz. While the low-frequency bandwidth extension was not found to improve perceived quality, the method reduced dissimilarity with wideband speech.

An experimental comparison of fundamental frequency tracking algorithms

“Yet another Algorithm for Pitch Tracking -YAAPT” was published in a 2010 JASA paper (Zahorian and Hu). Although demonstrated to provide high accuracy and noise robustness for fundamental frequency tracking for both studio quality speech and telephone speech, especially as compared to other well-known algorithms (YIN, Praat, RAPT), YAAPT has not been widely used, possibly due to the difficulty of using it and uncertainty about its effectiveness for difficult conditions. Therefore, more work has been done to improve the algorithm and especially to improve its functionality and ease of use as MATLAB functions. In the present paper, the current version of YAAPT is presented, along with clear documentation for using it, both stand alone and as a function to be called by another program. Experimentally, YAAPT is compared with YIN, Praat, RAPT, and a cepstrum method for studio bandwidth speech and telephone speech for a variety of noise conditions. Experiments are conducted with multiple databases, including American English, British English, and Mandarin Chinese. For most conditions evaluated, YAAPT gives better performance than the other fundamental frequency trackers.

Multitaper MFCC and PLP features for speaker verification using i-vectors

In this paper we study the performance of the low-variance multi-taper Mel-frequency cepstral coefficient (MFCC) and perceptual linear prediction (PLP) features in a state-of-the-art i-vector speaker verification system. The MFCC and PLP features are usually computed from a Hamming-windowed periodogram spectrum estimate. Such a single-tapered spectrum estimate has large variance, which can be reduced by averaging spectral estimates obtained using a set of different tapers, leading to a so-called multi-taper spectral estimate. The multi-taper spectrum estimation method has proven to be powerful especially when the spectrum of interest has a large dynamic range or varies rapidly. Multi-taper MFCC features were also recently studied in speaker verification with promising preliminary results. In this study our primary goal is to validate those findings using an up-to-date i-vector classifier on the latest NIST 2010 SRE data. In addition, we also propose to compute robust perceptual linear prediction (PLP) features using multitapers. Furthermore, we provide a detailed comparison between different taper weight selections in the Thomson multi-taper method in the context of speaker verification. Speaker verification results on the telephone (det5) and microphone speech (det1, det2, det3 and det4) of the latest NIST 2010 SRE corpus indicate that the multi-taper methods outperform the conventional periodogram technique. Instead of simply averaging (using uniform weights) the individual spectral estimates in forming the multi-taper estimate, weighted averaging (using non-uniform weights) improves performance. Compared to the MFCC and PLP baseline systems, the sine-weighted cepstrum estimator (SWCE) based multitaper method provides average relative reductions of 12.3% and 7.5% in equal error rate, respectively. For the multi-peak multi-taper method, the corresponding reductions are 12.6% and 11.6%, respectively. Finally, the Thomson multi-taper method provides error reductions of 9.5% and 5.0% in EER for MFCC and PLP features, respectively. We conclude that both the MFCC and PLP features computed via multitapers provide systematic improvements in recognition accuracy.

Conversational quality evaluation of artificial bandwidth extension of telephone speech

Artificial bandwidth extension methods have been developed to improve the quality and intelligibility of narrowband telephone speech and to reduce the difference with wideband speech. Such methods have commonly been evaluated with objective measures or subjective listening-only tests, but conversational evaluations have been rare. This article presents a conversational evaluation of two methods for the artificial bandwidth extension of telephone speech. Bandwidth-extended narrowband speech is compared with narrowband and wideband speech in a test setting including a simulated telephone connection, realistic conversation tasks, and various background noise conditions. The responses of the subjects indicate that speech processed with one of the methods is preferred to narrowband speech in noise, but wideband speech is superior to both narrowband and bandwidth-extended speech. Bandwidth extension was found to be beneficial for telephone conversation in noisy listening conditions.

Fine structure processing improves telephone speech perception in cochlear implant users

The objective of this study was to compare telephone speech perception and subjective preferences in cochlear implant users with two different speech-processing strategies: high-definition continuous interleaved sampling (HDCIS) and fine structure processing (FSP). A randomized double-blind study was designed for intra-individual comparison of HDCIS and FSP. Twenty-five post-lingually deafened patients with either the PulsarCI(100) or SonataTI(100) and Opus2 acoustic processor were tested consecutively with both coding strategies, assigned in a random order. Disyllabic word speech perception was tested 6 weeks after each fitting under the following conditions: landline use with (LWN) and without (LWoN) background noise, mobile use with (MWN), and without (MWoN) background noise and mobile use with a Bluetooth magnetic field transmitter necklace (MB). Changes in health-related quality of life (QoL) were assessed using the Glasgow Benefit Inventory (GBI) and Faber's questionnaire. Personal preferences between strategies were surveyed upon completion of the study. All subjects included in this study performed better with FSP in the landline tests. There was an improvement of 11.5 % in LWN use (p = 0.014; CI 95 % = 3-20 %) and 10 % in LWoN use (p = 0.001; CI 95 % = 5-15 %). MWoN showed an improvement of 6.3 % with FSP (p = 0.03; CI 95 % = 0-13 %). MB tests showed an improvement of 11 % with FSP (p < 0.05; CI 95 % = 1.5-22 %). Quality of life was significantly better using FSP. Eighty-four percent of participants preferred FSP. The FSP speech coding strategy improved the speech recognition of cochlear implant users when using the telephone compared to HDCIS. Cochlear implantation with FSP coding improved QoL.

Improved acoustic models for spontaneous speech recognition

This paper describes advances for acoustic models in Chinese spontaneous Conversational Telephone Speech (CTS) recognition task. A number of approaches were investigated in the acoustic modeling, including Heteroscedastic Linear Discriminant Analysis (HLDA), Vocal Tract Length Normalization (VTLN), Gaussianization, Minimum Phone Error (MPE), Feature space MPE (fMPE), and etc. Considering pronunciation variations in continuous speech, tones in recognition vocabulary were modified due to the Sandhi rule. The acoustic models were trained on over 200 hours of audio data from standard LDC corpora. The improved acoustic models reduce the relative Character Error Rate (CER) by about 25% over the baseline acoustic models on standard LDC test set and China 863 program evaluation data set. Acknowledgment: This work is partially supported by the National Natural Science Foundation of China (No's. 10925419, 90920302, 10874203, 60875014, 61072124, 11074275, 11161140319).