Small Vocabulary Speech Recognition Research Articles

Audio to Sign Language Translator

Abstract: This project is based on converting the audio signals receiver to text using speech to text API. Speech to text conversion comprises of small, medium and large vocabulary conversions. Such systems process or accept the voice which then gets converted to their respective text. This paper gives a comparative analysis of the technologies used in small, medium, and large vocabulary Speech Recognition System. The comparative study determines the benefits and liabilities of all the approaches so far. The experiment shows the role of language models in improving the accuracy of speech to text conversion system. We experiment the speech data with noisy sentences and incomplete words. The results show a prominent result for randomly chosen sentences compared to sequential set of sentences

Keyword Search Based on Unsupervised Pre-Trained Acoustic Models

Speech keyword search (KWS) is the task of automatically detecting the required keywords in continuous speech. Single-keyword detection can be regarded as the task of speech keyword wake-up. For many practical applications of these small vocabulary speech recognition tasks, it is costly and unnecessary to build a full large vocabulary speech recognition system. For tasks related to speech keyword search, insufficiency in data resources remains the main challenge so far. Speech pre-training has become an effective technique, showing its superiority in a variety of tasks. The key idea is to learn effective representations in settings where a large amount of unlabeled data is available to improve the performance while labeled data of downstream tasks are limited. This research focuses on the combination of unsupervised pre-training and keyword search based on the Keyword-Filler model and introduces unsupervised pre-training into speech keyword search. The research selects pre-trained model architecture Wav2vec2.0 including XLSR. The research results show that training with feature extracted by pre-trained model performs better than the baseline. In the case of low-resource condition, the baseline performance drops significantly, while the performance of the pre-trained tuned model does not decrease but even increases slightly in some intervals. It can be seen that the pre-trained model can be tuned to achieve better performance on very little data. This shows the advantage and application value of keyword search based on unsupervised pre-training.

Acoustic model topology optimization for large vocabulary speech recognition

Acoustic model topology selection work in constructing large vocabulary speech recognition systems is being done empirically or heuristically. In this paper, we propose two improved algorithms, which are based on Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) respectively, on the basis of our previously proposed algorithms to select and optimize model topologies for small or medium vocabulary speech recognition systems. Our improved algorithms attain the goal of optimizing acoustic model topologies for large vocabulary speech recognition systems mainly through modifying the encoding schemes of our previously proposed algorithms. Experiments on the dialogue corpus of Inner Mongolia University show that, compared with the conventional acoustic model topology selection method, our newly proposed algorithms are able to bring much higher recognition performance for large vocabulary speech recognition systems by optimizing their acoustic model topologies.

Fusion Feature Extraction Based on Auditory and Energy for Noise-Robust Speech Recognition

Environmental noise can pose a threat to the stable operation of current speech recognition systems. It is therefore essential to develop a front feature set that is able to identify speech under low signal-to-noise ratio. In this paper, a robust fusion feature is proposed that can fully characterize speech information. To obtain the cochlear filter cepstral coefficients (CFCC), a novel feature is first extracted by the power-law nonlinear function, which can simulate the auditory characteristics of the human ear. Speech enhancement technology is then introduced into the front end of feature extraction, and the extracted feature and their first-order difference are combined in new mixed features. An energy feature Teager energy operator cepstral coefficient (TEOCC) is also extracted, and combined with the above-mentioned mixed features to form the fusion feature sets. Principal component analysis (PCA) is then applied to feature selection and optimization of the feature set, and the final feature set is used in a non-specific persons, isolated words, and small-vocabulary speech recognition system. Finally, a comparative experiment of speech recognition is designed to verify the advantages of the proposed feature set using a support vector machine (SVM). The experimental results show that the proposed feature set not only display a high recognition rate and excellent anti-noise performance in speech recognition, but can also fully characterize the auditory and energy information in the speech signals.

Acoustic modelling for speech recognition in Indian languages in an agricultural commodities task domain

In developing speech recognition based services for any task domain, it is necessary to account for the support of an increasing number of languages over the life of the service. This paper considers a small vocabulary speech recognition task in multiple Indian languages. To configure a multi-lingual system in this task domain, an experimental study is presented using data from two linguistically similar languages – Hindi and Marathi. We do so by training a subspace Gaussian mixture model (SGMM) (Povey et al., 2011; Rose et al., 2011) under a multi-lingual scenario (Burget et al., 2010; Mohan et al., 2012a). Speech data was collected from the targeted user population to develop spoken dialogue systems in an agricultural commodities task domain for this experimental study. It is well known that acoustic, channel and environmental mismatch between data sets from multiple languages is an issue while building multi-lingual systems of this nature. As a result, we use a cross-corpus acoustic normalization procedure which is a variant of speaker adaptive training (SAT) (Mohan et al., 2012a). The resulting multi-lingual system provides the best speech recognition performance for both languages. Further, the effect of sharing “similar” context-dependent states from the Marathi language on the Hindi speech recognition performance is presented.

Using morphemes in language modeling and automatic speech recognition of Amharic

AbstractThis paper presents morpheme-based language models developed for Amharic (a morphologically rich Semitic language) and their application to a speech recognition task. A substantial reduction in the out of vocabulary rate has been observed as a result of using subwords or morphemes. Thus a severe problem of morphologically rich languages has been addressed. Moreover, lower perplexity values have been obtained with morpheme-based language models than with word-based models. However, when comparing the quality based on the probability assigned to the test sets, word-based models seem to fare better. We have studied the utility of morpheme-based language models in speech recognition systems and found that the performance of a relatively small vocabulary (5k) speech recognition system improved significantly as a result of using morphemes as language modeling and dictionary units. However, as the size of the vocabulary increases (20k or more) the morpheme-based systems suffer from acoustic confusability and did not achieve a significant improvement over a word-based system with an equivalent vocabulary size even with the use of higher order (quadrogram) n-gram language models.

Multilingual Italian – Lithuanian Small Vocabulary Speech Recognition via Selection of Phonetic Transcriptions

The amount of resources required to incorporate a new language into an ASR system is extensive and it is usually hard to provide more than one language per system. Naturally some research groups started developing the multilingual speech recognition principles. Opposite to most of the other researchers, in this work the original recognizer was not modified as we chose to use the phonetic “language” of the recognizer – the “transcriptions”, allowing the adaption of the original ASR engine to any language if enough acoustic resources are available. The results of our experiments proved that it is quite reasonable to use these methods to provide the multilingual speech interface for the user. It was established that the recognition accuracy can be acceptable for small vocabulary grammars (~93% for Lithuanian, and ~98% for Italian), satisfying the requirements of an IVR type of application where different small vocabulary grammars with a limited number of multilingual transcriptions can be loaded for each of the different stage of the HCI voice dialog, allowing the modeling of multilingual regional self-services. DOI: http://dx.doi.org/10.5755/j01.eee.121.5.1145

Small-vocabulary speech recognition using a silent speech interface based on magnetic sensing

This paper reports on word recognition experiments using a silent speech interface based on magnetic sensing of articulator movements. A magnetic field was generated by permanent magnet pellets fixed to relevant speech articulators. Magnetic field sensors mounted on a wearable frame measured the fluctuations of the magnetic field during speech articulation. These sensor data were used in place of conventional acoustic features for the training of hidden Markov models. Both small vocabulary isolated word recognition and connected digit recognition experiments are presented. Their results demonstrate the ability of the system to capture phonetic detail at a level that is surprising for a device without any direct access to voicing information.

Speech Recognition Approach Based on Speech Feature Clustering and HMM

The paper presents a Segment-Mean method for reducing the dimension of the speech feature parameters. K-Means function is used to group the speech feature parameters whose dimension has been reduced. And then the speech samples are classified into different clusters according to their features. It proposes a cross-group training algorithm for the speech feature parameters clustering which improves the accuracy of the clustering function. When recognizing speech, the system uses a cross-group HMM models algorithm to match patterns which reduces the calculation by more than 50% and without reducing the recognition rate of the small vocabulary speech recognition system.

Multi-stream speech recognition based on Dempster–Shafer combination rule

This paper aims at investigating the use of Dempster–Shafer (DS) combination rule for multi-stream automatic speech recognition. The DS combination is based on a generalization of the conventional Bayesian framework. The main motivation for this work is the similarity between the DS combination and findings of Fletcher on human speech recognition. Experiments are based on the combination of several Multi Layer Perceptron (MLP) classifiers trained on different representations of the speech signal. The TANDEM framework is adopted in order to use the MLP outputs into conventional speech recognition systems. We exhaustively investigate several methods for applying the DS combination into multi-stream ASR. Experiments are run on small and large vocabulary speech recognition tasks and aim at comparing the proposed technique with other frame-based combination rules (e.g. inverse entropy). Results reveal that the proposed method outperforms conventional combination rules in both tasks. Furthermore we verify that the performance of the combined feature stream is never inferior to the performance of the best individual feature stream. We conclude the paper discussing other applications of the DS combination and possible extensions.

An Automatic Lipreading System for Spoken Digits With Limited Training Data

It is well known that visual cues of lip movement contain important speech relevant information. This paper presents an automatic lipreading system for small vocabulary speech recognition tasks. Using the lip segmentation and modeling techniques we developed earlier, we obtain a visual feature vector composed of outer and inner mouth features from the lip image sequence for recognition. A spline representation is employed to transform the discrete-time sampled features from the video frames into the continuous domain. The spline coefficients in the same word class are constrained to have similar expression and are estimated from the training data by the EM algorithm. For the multiple-speaker/speaker-independent recognition task, an adaptive multimodel approach is proposed to handle the variations caused by various talking styles. After building the appropriate word models from the spline coefficients, a maximum likelihood classification approach is taken for the recognition. Lip image sequences of English digits from 0 to 9 have been collected for the recognition test. Two widely used classification methods, HMM and RDA, have been adopted for comparison and the results demonstrate that the proposed algorithm deliver the best performance among these methods.

Small-vocabulary speech recognition using surface electromyography

We present results of electromyographic (EMG) speech recognition on a small vocabulary of 15 English words. EMG speech recognition holds promise for mitigating the effects of high acoustic noise on speech intelligibility in communication systems, including those used by first responders (a focus of this work). We collected 150 examples per word of single-channel EMG data from a male subject, speaking normally while wearing a firefighter’s self-contained breathing apparatus. The signal processing consisted of an activity detector, a feature extractor, and a neural network classifier. Testing produced an overall average correct classification rate on the 15 words of 74% with a 95% confidence interval of (71%, 77%). Once trained, the subject used a classifier as part of a real-time system to communicate to a cellular phone and to control a robotic device. These tasks were performed under an ambient noise level of approximately 95 decibels. We also describe ongoing work on phoneme-level EMG speech recognition.

Speech Recognition as an Input to Electronic Assistive Technology

Automatic speech recognition systems can be enormously beneficial for people with physical disabilities, having the potential to provide a fast and easy-to-use input for computer access and control of the home environment. An overview, based on a review of the research literature and clinical observations, is given of the suitability and performance of speech recognition for computer access in two situations ± for people with no speech problems and for people with dysarthria ± and case studies are presented as examples. Commercial large vocabulary systems are often appropriate for people with no, mild or moderate speech problems, although adequate time, training and support are essential. Those with severe dysarthria achieve lower recognition rates, but speech recognition can still be a useful method of input for some individuals. Speech input as a means of controlling the home environment is mainly based on small vocabulary speech recognition, and recognition reliability is dependent upon the consistency of the speech of the individual. Although speech recognition may be a solution for some people, others reject the technology because of inadequate reliability. This is particularly a problem for people with dysarthria, where speech recognition rates can be unacceptably low due to inconsistent speech.

An investigation of continuous speech recognition performance from unconstrained speech utterances

This paper is concerned with small vocabulary speech recognition from conversational utterances over the telephone network. Modeling techniques are investigated for dealing with large numbers of nonvocabulary words and artifacts that arise in these utterances. A hidden Markov model (HMM)-based continuous speech recognition system using a frame synchronous Viterbi beam search decoder is used for recognition. Keyword models compete in the finite state network with ‘‘filler’’ models of nonkeyword speech. Several issues were investigated relating to the quality of acoustic representations and language representations for this task. The first issue that was investigated was the definition of acoustic subword units using allophone clustering procedures. The second issue was the size of the vocabulary used for modeling non-keyword utterances. Finally, the last issue was the use of language models in unconstrained speech tasks. Experimental results will be presented for a 20-keyword recognition task where performance was evaluated on continuous utterances from 22 speakers. The results showed that all of the procedures including decision tree based allophone clustering, better out-of-vocabulary speech representations, and language models contributed to overall recognition performance. The best performing system provided 76% average probability of keyword detection at 5.8 false alarms per keyword per hour.