Automatic Speech Recognition Component Research Articles

Pseudo-Phoneme Label Loss for Text-Independent Speaker Verification

Compared with text-independent speaker verification (TI-SV) systems, text-dependent speaker verification (TD-SV) counterparts often have better performance for their efficient utilization of speech content information. On this account, some TI-SV methods tried to boost performance by incorporating an extra automatic speech recognition (ASR) component to explore content information, such as c-vector. However, the introduced ASR component requires a large amount of annotated data and consumes high computation resources. In this paper, we propose a pseudo-phoneme label (PPL) loss for the TI-SR task by integrating content cluster loss at the frame level and speaker recognition loss at the segment level in a unified network by multitask learning, without additional data requirement and exhausting computation. By referring to HuBERT, we generate pseudo-phoneme labels to adjust a frame level feature distribution by deep cluster to ensure each cluster corresponds to an implicit pronunciation unit in the feature space. We compare the proposed loss with the softmax loss, center loss, triplet loss, log-likelihood-ratio cost loss, additive margin softmax loss and additive angular margin loss on the VoxCeleb database. Experimental results demonstrate the effectiveness of our proposed method.

Unified Architecture for Multichannel End-to-End Speech Recognition With Neural Beamforming

This paper proposes a unified architecture for end-to-end automatic speech recognition (ASR) to encompass microphone-array signal processing such as a state-of-the-art neural beamformer within the end-to-end framework. Recently, the end-to-end ASR paradigm has attracted great research interest as an alternative to conventional hybrid paradigms with deep neural networks and hidden Markov models. Using this novel paradigm, we simplify ASR architecture by integrating such ASR components as acoustic, phonetic, and language models with a single neural network and optimize the overall components for the end-to-end ASR objective: generating a correct label sequence. Although most existing end-to-end frameworks have mainly focused on ASR in clean environments, our aim is to build more realistic end-to-end systems in noisy environments. To handle such challenging noisy ASR tasks, we study multichannel end-to-end ASR architecture, which directly converts multichannel speech signal to text through speech enhancement. This architecture allows speech enhancement and ASR components to be jointly optimized to improve the end-to-end ASR objective and leads to an end-to-end framework that works well in the presence of strong background noise. We elaborate the effectiveness of our proposed method on the multichannel ASR benchmarks in noisy environments (CHiME-4 and AMI). The experimental results show that our proposed multichannel end-to-end system obtained performance gains over the conventional end-to-end baseline with enhanced inputs from a delay-and-sum beamformer (i.e., BeamformIT) in terms of character error rate. In addition, further analysis shows that our neural beamformer, which is optimized only with the end-to-end ASR objective, successfully learned a noise suppression function.

The CARES corpus: a database of older adult actor simulated emergency dialogue for developing a personal emergency response system

There has been limited research on automatic speech recognition systems developed specifically for older adults and there exist few older adult speech corpora available for training them. For our research, samples of primarily older adult voices within an emergency context were needed to help develop, train, and test the automatic speech recognition component of a novel, intelligent, speech-based personal emergency response system. We were unable to locate an existing speech corpus with all the properties we required. Specifically, these properties included spoken Canadian English, both male and female adult (especially older adult) speech, emotional or stressed speech, and emergency type dialogue. As a result, we created the Canadian adult regular and emergency speech (CARES) corpus. The goal of this paper will be to describe the design and development of the CARES corpus. The CARES corpus has been designed using information obtained from live emergency call centre call transcripts and research literature in the field of automatic speech recognition. This corpus consists of a collection of spontaneous speech, read sentences, simulated expression of words, phrases, and emergency scenarios from adult actors aged 23---91 years. The emphasis is on emergency type dialogue and older adult speech. A total of 40 participant voices are included in the corpus and over 70 % of the voices are from adults over the age of 50 years. Approximately 3,200 minutes of speech was acquired in total.

Performance Analysis and Improvement of Turkish Broadcast News Retrieval

This paper presents our work on the retrieval of spoken information in Turkish. Traditional speech retrieval systems perform indexing and retrieval over automatic speech recognition (ASR) transcripts, which include errors either because of out-of-vocabulary (OOV) words or ASR inaccuracy. We use subword units as recognition and indexing units to reduce the OOV rate and index alternative recognition hypotheses to handle ASR errors. Performance of such methods is evaluated on our Turkish Broadcast News Corpus with two types of speech retrieval systems: a spoken term detection (STD) and a spoken document retrieval (SDR) system. To evaluate the SDR system, we also build a spoken information retrieval (IR) collection, which is the first for Turkish. Experiments showed that word segmentation algorithms are quite useful for both tasks. SDR performance is observed to be less dependent on the ASR component, whereas any performance change in ASR directly affects STD. We also present extensive analysis of retrieval performance depending on query length, and propose length-based index combination and thresholding strategies for the STD task. Finally, a new approach, which depends on the detection of stems instead of complete terms, is tried for STD and observed to give promising results. Although evaluations were performed in Turkish, we expect the proposed methods to be effective for similar languages as well.

The effectiveness of computer assisted pronunciation training for foreign language learning by children

This study investigates whether a computer assisted pronunciation training (CAPT) system can help young learners improve word-level pronunciation skills in English as a foreign language at a level comparable to that achieved through traditional teacher-led training. The pronunciation improvement of a group of learners of 11 years of age receiving teacher-fronted instruction was compared to that of a group receiving computer assisted pronunciation training by means of a system including an automatic speech recognition component. Results show that 1) pronunciation quality of isolated words improved significantly for both groups of subjects, and 2) both groups significantly improved in pronunciation quality of words that were considered particularly difficult to pronounce and that were likely to have been unknown to them prior to the training. Training with a computer-assisted pronunciation training system with a simple automatic speech recognition component can thus lead to short-term improvements in pronunciation that are comparable to those achieved by means of more traditional, teacher-led pronunciation training.

Spoken language understanding

Semantics deals with the organization of meanings and the relations between sensory signs or symbols and what they denote or mean. Computational semantics performs a conceptualization of the world using computational processes for composing a meaning representation structure from available signs and their features present, for example, in words and sentences. Spoken language understanding (SLU) is the interpretation of signs conveyed by a speech signal. SLU and natural language understanding (NLU) share the goal of obtaining a conceptual representation of natural language sentences. Specific to SLU is the fact that signs to be used for interpretation are coded into signals along with other information such as speaker identity. Furthermore, spoken sentences often do not follow the grammar of a language; they exhibit self-corrections, hesitations, repetitions, and other irregular phenomena. SLU systems contain an automatic speech recognition (ASR) component and must be robust to noise due to the spontaneous nature of spoken language and the errors introduced by ASR. Moreover, ASR components output a stream of words with no structure information like punctuation and sentence boundaries. Therefore, SLU systems cannot rely on such markers and must perform text segmentation and understanding at the same time.

Talking to Machines: Introducing Robot Perception to Resolve Speech Recognition Uncertainties

The use of spontaneous speech as a form of communication between humans and robots is a potential solution for more efficient human-robot interactions. Accuracy is one of the main problems associated with the automatic speech recognition (ASR) component of human-robot interactive systems. The standard ASR approach is based on statistical methods applied to phoneme domains. However, some problems cannot be solved with the rule-based approaches used so far; therefore, alternative strategies could be the solution. The aim of this paper is to investigate some aspects related to the use of a robot's perceptive abilities to increase the robustness of ASR components. The robot evaluative abilities are used to incrementally build knowledge that will be used during the recognition phase. This paper covers aspects concerning the use of time-warping algorithms to improve the speech recognition performance. In particular, aspects related to the accuracy and efficiency of this approach when applied to whole-sentence speech signals are discussed.