Automatic Recognition Of Continuous Speech Research Articles

Real-Time Automatic Continuous Speech Recognition System for Kannada Language/Dialects

Real-Time Automatic Continuous Speech Recognition System for Kannada Language/Dialects

Perspectivas sobre la traducción automática del habla

En Japón se han llevado a cabo muchas actividades de investigación acerca de la traducción automática del habla. Este artículo pretende ofrecer una visión general de dichas actividades y presentar las que se han realizado más recientemente. El sistema S2ST está formado básicamente por tres módulos: el reconocimiento automático del habla continua y de amplios vocabularios (Automatic Speech Recognition, ASR), la traducción automática de textos (Machine translation, MT) y la conversión de texto a voz (Text-to-Speech Synthesis, TTS). Todos los módulos deben ser plurilingües, por lo cual se requieren discursos y corpus multilingües para los modelos de formación. El rendimiento del sistema S2ST mejora considerablemente por medio de un aprendizaje profundo y grandes corpus formativos. Sin embargo, todavía hace falta tratar diversos aspectos, com la simultaneidad, la paralingüística, la dependencia del contexto y de la situación, la intención y la dependencia cultural. Por todo ello, repasaremos las actividades de investigación actuales y discutiremos varias cuestiones relacionadas con la traducción automática del habla de última generación.

Robust automatic continuous speech recognition for 'Adi', a zero-resource indigenous language of Arunachal Pradesh

Robust automatic continuous speech recognition for 'Adi', a zero-resource indigenous language of Arunachal Pradesh

Performance Analysis of various Front-end and Back End Amalgamations for Noise-robust DNN-based ASR

Introduction: An Automatic Speech Recognition (ASR) system enables to recognize the speech utterances and thus can be used to convert speech into text for various purposes. These systems are deployed in different environments such as clean or noisy and are used by all ages or types of people. These also present some of the major difficulties faced in the development of an ASR system. Thus, an ASR system need to be efficient, while also being accurate and robust. Our main goal is to minimize the error rate during training as well as testing phases, while implementing an ASR system. Performance of ASR depends upon different combinations of feature extraction techniques and back-end techniques. In this paper, using a continuous speech recognition system, the performance comparison of different combinations of feature extraction techniques and various types of back-end techniques has been presented Methods: Hidden Markov Models (HMMs), Subspace Gaussian Mixture Models (SGMMs) and Deep Neural Networks (DNNs) with DNN-HMM architecture, namely Karel’s, Dan’s and Hybrid DNN-SGMM architecture are used at the back-end of the implemented system. Mel frequency Cepstral Coefficient (MFCC), Perceptual Linear Prediction (PLP), and Gammatone Frequency Cepstral coefficients (GFCC) are used as feature extraction techniques at the front-end of the proposed system. Kaldi toolkit has been used for the implementation of the proposed work. The system is trained on the Texas Instruments-Massachusetts Institute of Technology (TIMIT) speech corpus for English language Results: The experimental results show that MFCC outperforms GFCC and PLP in noiseless conditions, while PLP tends to outperform MFCC and GFCC in noisy conditions. Furthermore, the hybrid of Dan’s DNN implementation along with SGMM performs the best for the back-end acoustic modeling. The proposed architecture with PLP feature extraction technique in the front end and hybrid of Dan’s DNN implementation along with SGMM at the back end outperforms the other combinations in a noisy environment. Conclusion: Automatic Speech recognition has numerous applications in our lives like Home automation, Personal assistant, Robotics etc. It is highly desirable to build an ASR system with good performance. The performance Automatic Speech Recognition is affected by various factors which include vocabulary size, whether system is speaker dependent or independent, whether speech is isolated, discontinuous or continuous, adverse conditions like noise. The paper presented an ensemble architecture that uses PLP for feature extraction at the front end and a hybrid of SGMM + Dan’s DNN in the backend to build a noise robust ASR system Discussion: The presented work in this paper discusses the performance comparison of continuous ASR systems developed using different combinations of front-end feature extraction (MFCC, PLP, and GFCC) and back-end acoustic modeling (mono-phone, tri-phone, SGMM, DNN and hybrid DNN-SGMM) techniques. Each type of front-end technique is tested in combination with each type of back-end technique. Finally, it compares the results of the combinations thus formed, to find out the best performing combination in noisy and clean conditions

Streaming cascade-based speech translation leveraged by a direct segmentation model

The cascade approach to Speech Translation (ST) is based on a pipeline that concatenates an Automatic Speech Recognition (ASR) system followed by a Machine Translation (MT) system. Nowadays, state-of-the-art ST systems are populated with deep neural networks that are conceived to work in an offline setup in which the audio input to be translated is fully available in advance. However, a streaming setup defines a completely different picture, in which an unbounded audio input gradually becomes available and at the same time the translation needs to be generated under real-time constraints. In this work, we present a state-of-the-art streaming ST system in which neural-based models integrated in the ASR and MT components are carefully adapted in terms of their training and decoding procedures in order to run under a streaming setup. In addition, a direct segmentation model that adapts the continuous ASR output to the capacity of simultaneous MT systems trained at the sentence level is introduced to guarantee low latency while preserving the translation quality of the complete ST system. The resulting ST system is thoroughly evaluated on the real-life streaming Europarl-ST benchmark to gauge the trade-off between quality and latency for each component individually as well as for the complete ST system.

Automatic Continuous Russian Speech Recognition system based on deep neural networksl

Automatic Continuous Russian Speech Recognition system based on deep neural networksl

System-independent ASR error detection and classification using Recurrent Neural Network

This paper addresses errors in continuous Automatic Speech Recognition (ASR) in two stages: error detection and error type classification. Unlike the majority of research in this field, we propose to handle the recognition errors independently from the ASR decoder. We first establish an effective set of generic features derived exclusively from the recognizer output to compensate for the absence of ASR decoder information. Then, we apply a variant Recurrent Neural Network (V-RNN) based models for error detection and error type classification. Such model learn additional information to the recognized word classification using label dependency. As a result, experiments on Multi-Genre Broadcast Media corpus have shown that the proposed generic features setup leads to achieve competitive performances, compared to state of the art systems in both tasks. Furthermore, we have shown that V-RNN trained on the proposed feature set appear to be an effective classifier for the ASR error detection with an Accuracy of 85.43%.

GFCC based discriminatively trained noise robust continuous ASR system for Hindi language

A statistically designed Automatic Speech Recognition (ASR) system extracts features from speech signals using feature extraction methods, links the extracted features with the expected phonetics of the hypothesis using acoustic models, uses language model to add prior information about the structure of the target language. For many years, Mel-frequency Cepstral coefficients (MFCC), n-gram, and Hidden Markov Model (HMM) approaches have been used predominantly for feature extraction, language modeling and acoustic modeling, respectively. However, performance degradation of MFCC in noisy conditions and inaccuracy of HMMs while handling large vocabularies have made researchers to propose more efficient methods. The proposed work uses noise robust method Gammatone Frequency Cepstral Coefficients(GFCC) for feature extraction, trigram language modeling, and HMM-Gaussian mixture model (GMM) based acoustic modeling to implement a continuous Hindi language ASR system. Also, it applies Differential Evolution (DE) technique to refine the GFCC features and discriminative techniques to enhance performance of the acoustic model. The performance of the implemented system has been evaluated by using different feature extraction methods, variants of n-gram language modeling techniques and different discriminative techniques in clean as well as noisy conditions. Initially, the results reveal that DE optimized GFCC with HMM-Gaussian Mixture Model (GMM) acoustic modeling performs better than MFCC, PLP and MF-PLP feature extraction methods. Secondly, the experimental results show that the Minimum Phone Error (MPE) outperforms Maximum Mutual Information (MMI) and Maximum Likelihood Estimation (MLE) and trigram based language modeling gives more accurate results than unigram and bigram language modeling. Finally, it has been concluded that the continuous Hindi language ASR system implemented using DE refined GFFC feature extraction method with MPE discriminative training technique and trigram based language modeling gives better accuracy in clean as-well-as noisy environments.

A study of neural network Russian language models for automatic continuous speech recognition systems

We show the results of studying models of the Russian language constructed with recurrent artificial neural networks for systems of automatic recognition of continuous speech. We construct neural network models with different number of elements in the hidden layer and perform linear interpolation of neural network models with the baseline trigram language model. The resulting models were used at the stage of rescoring the N best list. In our experiments on the recognition of continuous Russian speech with extra-large vocabulary (150 thousands of word forms), the relative reduction in the word error rate obtained after rescoring the 50 best list with the neural network language models interpolated with the trigram model was 14%.

Speech translation system for english to dravidian languages

In this paper the Speech-to-Speech Translation (SST) system, which is mainly focused on translation from English to Dravidian languages (Tamil and Malayalam) has been proposed. Three major techniques involved in SST system are Automatic continuous speech recognition, machine translation, and text-to-speech synthesis system. In this paper automatic Continuous Speech Recognition (CSR) has been developed based on the Auto Associative Neural Network (AANN), Support Vector Machine (SVM) and Hidden Markov Model (HMM). The HMM yields better results compared with SVM and AANN. Hence the HMM based Speech recognizer for English language has been taken. We propose a hybrid Machine Translation (MT) system (combination of Rule based and Statistical) for converting English to Dravidian languages text. A syllable based concatenative Text To Speech Synthesis (TTS) for Tamil and Malayalam has been proposed. AANN based prosody prediction has been done for the Tamil language which is used to improve the naturalness and intelligibility. The domain is restricted to sentences that cover the announcements in the railway station, bus stop and airport. This work is framed a novel translation method for English to Dravidian languages. The improved performance of each module HMM based CSR, Hybrid MT and concatenative TTS increases the overall speech translation performance. This proposed speech translation system can be applied to English to any Indian languages if we train and create a parallel corpus for those languages.

Acoustic classification and segmentation using modified spectral roll-off and variance-based features

This paper presents novel features and an architecture for an automatic on-line acoustic classification and segmentation system. The system includes speech/non-speech segmentation (with the emphasis on accurate speech/music segmentation), gender segmentation, and speech bandwidth segmentation. This automatic segmentation system can be easily integrated into an automatic continuous speech recognition system, where information about individual acoustic segments can be used for acoustic model selection and adaptation, or as additional information for rich transcription output. Acoustic model adaptation can improve the speech recognition rate and additional information in rich transcription can be useful when searching for some certain events or circumstances (male speaker talking over the phone line, etc.). For speech/non-speech segmentation we propose a new set of features, which are based on an energy variance in a narrow frequency sub-band, called EVFB (Energy Variance of Filter Bank). The proposed features also prove to be an efficient discriminator between speech and music. Segmentation cross-test results show that EVFB features prove to be more robust than MFCC features. Two new features (modified spectral roll-off and high-frequency energy variance) are also proposed for speech bandwidth classification and segmentation. The results show a good and robust performance by the automatic on-line acoustic segmentation system. All experiments and tests were performed on a radio broadcast database and a Slovenian BNSI Broadcast News database. Integration of the automatic on-line acoustic segmentation system into a continuous speech recognition system based on MFCC (mel-frequency cepstral coefficients) features requires only a small additional computational cost because many of the proposed systemʼs feature calculation procedures are common to the MFCC features calculation procedure. ► Article presents a system for on-line acoustic classification and segmentation. ► The system supports speech/non-speech, gender, and bandwidth segmentation. ► For speech/non-speech and speech bandwidth segmentation novel features are presented. ► The novel EVFB features also prove to be a good speech/music discriminator.

Within-word pronunciation variation modeling for Arabic ASRs: a direct data-driven approach

Pronunciation variation is a major obstacle in improving the performance of Arabic automatic continuous speech recognition systems. This phenomenon alters the pronunciation spelling of words beyond their listed forms in the pronunciation dictionary, leading to a number of out of vocabulary word forms. This paper presents a direct data-driven approach to model within-word pronunciation variations, in which the pronunciation variants are distilled from the training speech corpus. The proposed method consists of performing phoneme recognition, followed by a sequence alignment between the observation phonemes generated by the phoneme recognizer and the reference phonemes obtained from the pronunciation dictionary. The unique collected variants are then added to dictionary as well as to the language model. We started with a Baseline Arabic speech recognition system based on Sphinx3 engine. The Baseline system is based on a 5.4 hours speech corpus of modern standard Arabic broadcast news, with a pronunciation dictionary of 14,234 canonical pronunciations. The Baseline system achieves a word error rate of 13.39%. Our results show that while the expanded dictionary alone did not add appreciable improvements, the word error rate is significantly reduced by 2.22% when the variants are represented within the language model.

Toward enhanced Arabic speech recognition using part of speech tagging

One major source of suboptimal performance in automatic continuous speech recognition systems is misrecognition of small words. In general, errors resulting from small words are much more than errors resulting from long words. Therefore, compounding some words (small or long) to produce longer words is welcome by speech recognition decoders. In this paper, we present a novel approach to artificially generate compound words using part of speech tagging. For this purpose, we consider two cases in Arabic speech where two words are pronounced without a silence period in between: a noun followed by an adjective, and a preposition followed by any word. To collect the candidate compound words, we use Stanford Arabic tagger to tag all words in our baseline transcription corpus. Then, compound words are generated whenever any of the two cases occur in a sequence of two words. The unique compound words are then added to the expanded pronunciation dictionary, as well as to the language model. Using Sphinx 3, we test the proposed method for a 5.4 hours speech corpus of modern standard Arabic. The results show a significant improvement, as the word error rate is reduced by 2.39%.

Modern standard Arabic speech corpus for implementing and evaluating automatic continuous speech recognition systems

This paper presents our work towards developing a new speech corpus for Modern Standard Arabic (MSA), which can be used for implementing and evaluating Arabic speaker-independent, large vocabulary, automatic, and continuous speech recognition systems. The speech corpus was recorded by 40 (20 male and 20 female) Arabic native speakers from 11 countries representing three major regions (Levant, Gulf, and Africa). Three development phases were conducted based on the size of training data, Gaussian mixture distributions, and tied states (senones). Based on our third development phase using 11 hours of training speech data, the acoustic model is composed of 16 Gaussian mixture distributions and the state distributions tied to 300 senones. Using three different data sets, the third development phase obtained 94.32% and 8.10% average word recognition correctness rate and average Word Error Rate (WER), respectively, for same speakers with different sentences (testing sentences). For different speakers with same sentences (training sentences), this work obtained 98.10% and 2.67% average word recognition correctness rate and average WER, respectively, whereas for different speakers with different sentences (testing sentences) this work obtained 93.73% and 8.75% average word recognition correctness rate and average WER, respectively.

Audio-Visual Speech Recognition Using MPEG-4 Compliant Visual Features

We describe an audio-visual automatic continuous speech recognition system, which significantly improves speech recognition performance over a wide range of acoustic noise levels, as well as under clean audio conditions. The system utilizes facial animation parameters (FAPs) supported by the MPEG-4 standard for the visual representation of speech. We also describe a robust and automatic algorithm we have developed to extract FAPs from visual data, which does not require hand labeling or extensive training procedures. The principal component analysis (PCA) was performed on the FAPs in order to decrease the dimensionality of the visual feature vectors, and the derived projection weights were used as visual features in the audio-visual automatic speech recognition (ASR) experiments. Both single-stream and multistream hidden Markov models (HMMs) were used to model the ASR system, integrate audio and visual information, and perform a relatively large vocabulary (approximately 1000 words) speech recognition experiments. The experiments performed use clean audio data and audio data corrupted by stationary white Gaussian noise at various SNRs. The proposed system reduces the word error rate (WER) by 20% to 23% relatively to audio-only speech recognition WERs, at various SNRs (0–30 dB) with additive white Gaussian noise, and by 19% relatively to audio-only speech recognition WER under clean audio conditions.

Multi-band automatic speech recognition

This paper presents a new architecture for automatic speech recognition systems which is characterized by the division of the spectral domain of the speech signal into several independent frequency bands. This model is based on the psycho-acoustic work of Fletcher (1953) who proposed a similar principle for the human auditory system. Jont B. Allen published a paper in 1994 in which he summarized the work of Fletcher and also proposed to adapt the multi-band paradigm to automatic speech recognition (ASR) (Allen, 1994). Many researchers have then studied this principle and built such ASR systems. The goal of this paper is to analyse some of the most important issues in the design of a multi-band ASR system in order to determine which architecture it should have in which environment. Two other major problems are then considered: how to train multi-band systems and how to use them for continuous ASR.

Use of fuzzy mathematical concepts in character spotting for automatic recognition of continuous speech in Hindi

In this paper we describe the use of fuzzy mathematical concepts in spotting characters from continuous speech in the Indian language Hindi. The research effort reported here is concerned with identifying the phonetic features from the acoustic parameters of a speech signal and combining the features to spot a character in continuous speech. These functions are performed by using expert systems that use confidence values for grading the conclusions arrived at each stage of processing. This paper discusses how this grading is obtained by using numerical translation of the description of a character obtained from an expert phonetician. Confidence values are assigned to the premises of the rules obtained from an expert phonetician using fuzzy membership functions. Fuzzy relations are used to grade the conclusions derived from these premises. Results of applying these techniques to character spotting are given.

Modeling word‐boundary phonological processes in continuous speech recognition

The automatic recognition of continuous English speech, as against isolated words, must contend with phonological processes at word boundaries that can distort the mapping between surface form and lexical entry. Past recognition systems have used generative phonological rules in a precompilation stage to expand the working dictionary. However, this greatly increases the bulk and complexity of the lexicon. It is also possible to use analytic rules to undo the putative effect of phonological processes at the time of recognition. However, this can lead to the postulation of nonwords and to slower processing times. A solution that blends the advantages of both approaches, with the disadvantages of neither, is to use finite state transducers (FSTs) as filters on permitted matchings between input strings and lexical entries. These were implemented in a recognition system, and found to increase the percentage of words recognized by three percentage points, at a cost of halving the signal‐to‐noise ratio, compared to the same system without phonological FSTs.

Benchmark test procedures for automatic speech recognition systems

Benchmark test procedures have been developed and implemented in the DARPA‐sponsored program in automatic recognition of continuous speech. These tests make use of a database of recorded speech that includes material for both speaker‐dependent and speaker‐independent technologies. In implementing these uniform tests, it is necessary to define a standard lexicon of units to be scored and a standard or reference orthographic representation for each sentence, and to select a scoring procedure and error taxonomy. Considerations taken in identifying the reference lexicon and representations will be outlined. For the DARPA program, a dynamic‐programming string alignment procedure was implemented, and errors were defined in terms of substitutions, insertions, and deletions. This scoring software is publicly available. Known limitations of this procedure will be discussed. [Work supported by DARPA.]

Word‐boundary identification in automatic continuous speech recognition

Phoneme sequence constraints, such as /b l g/, that occur across word boundaries (considerable growth) but not word internally were used for word‐boundary identification from continuous speech transcriptions. All three‐phoneme, word‐boundary sequences were derived by pairing P♯ (all word‐final phonemes) with ♯PP (all word‐initial, two‐phoneme sequences) and PP♯ with ♯P in a 23 000‐word lexicon including 70 000 reduced phonological forms. The resulting P♯PP and PP♯P sequences were matched against the same lexicon to determine which of them were excluded word internally. All boundary sequences excluded word internally were compiled into a tree structure that was matched against three sets of phonemic transcriptions corresponding to “fast,” “normal,” and “citation” speech production styles of 145 utterances. The results showed a word‐boundary detection rate of between 39% and 50% in the three transcription sets. The incorporation of phoneme sequence constraints also reduces the number of word parsings compared with an earlier left‐to‐right chart‐parsing approach for word‐boundary identification [J. M. Harrington and A. Johnstone, Comput. Speech Lang. (1988), in press].