Speech Recognition Applications Research Articles

Step forward to map fully parallel energy efficient cortical columns on field programmable gate arrays

This study presents energy and area-efficient hardware architectures to map fully parallel cortical columns on reconfigurable platform – field programmable gate arrays (FPGAs). An area-efficient architecture is proposed at the system level and benchmarked with a speech recognition application. Owing to the spatio-temporal nature of spiking neurons it is more suitable to map such architectures on FPGAs where signals can be represented in binary form and communication can be performed through the use of spikes. The viability of implementing multiple recurrent neural reservoirs is demonstrated with a novel multiplier-less reconfigurable architectures and a design strategy is devised for its implementation.

Skew Gaussian mixture models for speaker recognition

Gaussian mixture models (GMMs) are widely used in speech and speaker recognition. This study explores the idea that a mixture of skew Gaussians might capture better feature vectors that tend to have skew empirical distributions. It begins with deriving an expectation maximisation (EM) algorithm to train a mixture of two-piece skew Gaussians that turns out to be not much more complicated than the usual EM algorithm used to train symmetric GMMs. Next, the algorithm is used to compare skew and symmetric GMMs in some simple speaker recognition experiments that use Mel frequency cepstral coefficients (MFCC) and line spectral frequencies (LSF) as the feature vectors. MFCC are one of the most popular feature vectors in speech and speaker recognition applications. LSF were chosen because they exhibit significantly more skewed distribution than MFCC and because they are widely used [together with the related immittance spectral frequencies (ISF)] in speech transmission standards. In the reported experiments, models with skew Gaussians performed better than models with symmetric Gaussians and skew GMMs with LSF compared favourably with both skew symmetric and symmetric GMMs that used MFCC.

99.8 percent accuracy achieved on Peterson and Barney (1952) acoustic measurements

In 2012, a paper was presented (Reetz, 2012) discussing the lack of working phonemic models, which was an acknowledgment to an earlier presentation (Ladefoged, 2004) discussing 50 + years of phonetics and phonology. These presentations highlighted the successes in phonological research over the last 60 and 50 years, respectively, but both concluded that there is still no recognized working model of phoneme identification. This presentation will discuss the Waveform Model of Vowel Perception (Stokes, 2009) achieving 99.8% accuracy on the Peterson and Barney (1952) dataset using 30 conditional statements across all ten vowels produced by the 33 males (509/510 for the vowels identified by humans at 100%). These results replicate and improve on the 99.2% achieved across the vowels produced by the males in the Hillenbrand (1995) dataset (Stokes, 2011). As a logical progression, ELBOW was developed in 2013 using the algorithm developed for static data to identify streaming vowel productions achieving over 91% before introducing improvements. Beyond ELBOW, it was essential to replicate earlier results on the most cited dataset in the literature. The Waveform Model has now replicated human performance across multiple datasets and is being successfully introduced into automatic speech recognition applications.

Phoneme and Viseme based Approach for Lip Synchronization

Phoneme to Viseme mapping has great application in Visual Speech Recognition, Lip Synchronization, Talking Head Applications, movies, news reading, and film industries. Lot of work has been done in area of various face component detection and recognition. Apart from eye detection, ear detection, iris detection etc, lip tracking and lip detection is one of the favourite topics for researchers. Various algorithms and techniques have been implemented so far to achieve better and better performance. Normalized RGB colour scheme, HSV colour model, Lip detection using HUE segmentation and many more techniques have been implemented and are in the boom. All methods are having their own pros and cons. We are aiming to extract out phonemes from speech as well as we extract visual feature i.e. visemes from face by using hue and saturation values. The reason behind the selection of this algorithm is that, it performs well under various illumination conditions, which is the one of the dimension of difficulty in the area of lip detection. We are aiming to carry out the work on in-house database with varying lighting and noisy conditions.

Feature extraction technique using ERB like wavelet sub-band periodic and aperiodic decomposition for TIMIT phoneme recognition

In the recent years, wavelet transform has been found to be an effective tool for the time---frequency analysis for non-stationary and quasi-stationary signals such as speech signals. In the recent past, wavelet transform has been used as feature extraction in speech recognition applications. Here we propose a wavelet based feature extraction technique that signifies both the periodic and aperiodic information along with sub-band instantaneous frequency of speech signal for robust speech recognition in noisy environment. This technique is based on parallel distributed processing technique inspired by the human speech perception process. This frontend feature processing technique employs equivalent rectangular bandwidth (ERB) filter like wavelet speech feature extraction method called Wavelet ERB Sub-band based Periodicity and Aperiodicity Decomposition (WERB-SPADE), and examines its validity for TIMIT phone recognition task in noisy environments. The speech sound is filtered by 24 band ERB like wavelet filter banks, and then the equal loudness pre-emphasized output of each band is processed through comb filter. Each comb filter is designed individually for each frequency sub-band to decompose the signal into periodic and aperiodic features. Thus it takes the advantage of the robustness shown by periodic features without losing certain important information like formant transition incorporated in aperiodic features. Speech recognition experiments with a standard HMM recognizer under both clean-training and multi-training condition training is conducted. Proposed technique shows more robustness compared to other features especially in noisy condition.

Auditory ERB like admissible wavelet packet features for TIMIT phoneme recognition

In recent years wavelet transform has been found to be an effective tool for time–frequency analysis. Wavelet transform has been used as feature extraction in speech recognition applications and it has proved to be an effective technique for unvoiced phoneme classification. In this paper a new filter structure using admissible wavelet packet is analyzed for English phoneme recognition. These filters have the benefit of having frequency bands spacing similar to the auditory Equivalent Rectangular Bandwidth (ERB) scale. Central frequencies of ERB scale are equally distributed along the frequency response of human cochlea. A new sets of features are derived using wavelet packet transform's multi-resolution capabilities and found to be better than conventional features for unvoiced phoneme problems. Some of the noises from NOISEX-92 database has been used for preparing the artificial noisy database to test the robustness of wavelet based features.

Efficient data selection for speech recognition based on prior confidence estimation using speech and monophone models

This paper proposes an efficient speech data selection technique that can identify those data that will be well recognized. Conventional confidence measure techniques can also identify well-recognized speech data. However, those techniques require a lot of computation time for speech recognition processing to estimate confidence scores. Speech data with low confidence should not go through the time-consuming recognition process since they will yield erroneous spoken documents that will eventually be rejected. The proposed technique can select the speech data that will be acceptable for speech recognition applications. It rapidly selects speech data with high prior confidence based on acoustic likelihood values and using only speech and monophone models. Experiments show that the proposed confidence estimation technique is over 50 times faster than the conventional posterior confidence measure while providing equivalent data selection performance for speech recognition and spoken document retrieval.

An iterative longest matching segment approach to speech enhancement with additive noise and channel distortion

This paper presents a new approach to speech enhancement from single-channel measurements involving both noise and channel distortion (i.e., convolutional noise), and demonstrates its applications for robust speech recognition and for improving noisy speech quality. The approach is based on finding longest matching segments (LMS) from a corpus of clean, wideband speech. The approach adds three novel developments to our previous LMS research. First, we address the problem of channel distortion as well as additive noise. Second, we present an improved method for modeling noise for speech estimation. Third, we present an iterative algorithm which updates the noise and channel estimates of the corpus data model. In experiments using speech recognition as a test with the Aurora 4 database, the use of our enhancement approach as a preprocessor for feature extraction significantly improved the performance of a baseline recognition system. In another comparison against conventional enhancement algorithms, both the PESQ and the segmental SNR ratings of the LMS algorithm were superior to the other methods for noisy speech enhancement.

Admissible wavelet packet features based on human inner ear frequency response for Hindi consonant recognition

It was observed that for non-stationary and quasi-stationary signals, wavelet transform has been found to be an effective tool for the time–frequency analysis. In the recent years wavelet transform being used for feature extraction in speech recognition applications. Here a new filter structure using admissible wavelet packet analysis is proposed for Hindi phoneme recognition. These filters have the benefit of having frequency bands spacing similar to the auditory Equivalent Rectangular Bandwidth (ERB) scale whose central frequencies are equally distributed along the frequency response of human cochlea. The phoneme recognition performance of proposed feature is compared with the standard baseline features and 24-band admissible wavelet packet-based features using a Hidden Markov Model (HMM) based classifier. Proposed feature shows better performance compared to conventional features for Hindi consonant recognition. To evaluate the robustness of proposed feature in the noisy environment NOISEX-92 database has been used.

An Improved Feature Extraction Method for Malay Vowel Recognition based on Spectrum Delta

Malay speech recognition is becoming popular among Malaysian researchers. In Malaysia, more local researchers are focusing on noise robust and accurate independent speaker speech recognition systems that use Malay language. The performance of speech recognition application under adverse noisy condition often becomes the topic of interest among speech recognition researchers in any languages. This paper presents a study of noise robust capability of an improved vowel feature extraction method called Spectrum Delta (SpD). The features are extracted from both original data and noise-added data and classified using three classifiers; (i) Linear Discriminant Analysis (LDA), (ii) K-Nearest Neighbors (k-NN) and (iii) Multinomial Logistic Regression (MLR). Most of the dependent and independent speaker systems which use mostly multi-framed analysis, yielded accuracy between 89% to 100% for dependent speaker system and between 70% to 94% for an independent speaker. This study shows that SpD features obtained an accuracy of 92.42% to 95.11% using all the four classifiers on a single framed analysis which makes this result comparable to those analysed with multi-framed approach.

Quantile Acoustic Vectors vs. MFCC Applied to Speaker Verification

In this paper we describe speaker and command recognition related experiments, through quantile vectors and Gaussian Mixture Modelling (GMM). Over the past several years GMM and MFCC have become two of the dominant approaches for modelling speaker and speech recognition applications. However, memory and computational costs are important drawbacks, because autonomous systems suffer processing and power consumption constraints; thus, having a good trade-off between accuracy and computational requirements is mandatory. We decided to explore another approach (quantile vectors in several tasks) and a comparison with MFCC was made. Quantile acoustic vectors are proposed for speaker verification and command recognition tasks and the results showed very good recognition efficiency. This method offered a good trade-off between computation times, characteristics vector complexity and overall achieved efficiency.

Using Speech Recognition in Learning Primary School Mathematics via Explain, Instruct and Facilitate Techniques

The application of Information and Communication Technologies has transformed traditional Teaching and Learning in the past decade to computerized-based era. This evolution has resulted from the emergence of the digital system and has greatly impacted on the global education and socio-cultural development. Multimedia has been absorbed into the education sector for producing a new learning concept and a combination of educational and entertainment approach. This research is concerned with the application of Window Speech Recognition and Microsoft Visual Basic 2008 Integrated/Interactive Development Environment in Multimedia-Assisted Courseware prototype development for Primary School Mathematics contents, namely, single digits and the addition. The Teaching and Learning techniques—Explain, Instruct and Facilitate are proposed and these could be viewed as instructors’ centered strategy, instructors’—learners’ dual communication and learners' active participation. The prototype is called M-EIF and deployed only users' voices; hence the activation of Window Speech Recognition is required prior to a test run.

Hindi viseme recognition using subspace DCT features

Many automatic speech recognition (ASR) applications may have noisy background conditions; hence, robustness has become an important area of research. The automated recognition of human speech using features from the visual domain along with audio has proved to be useful under these conditions. In this paper, use of visual information is proposed to increase the recognition performance and robustness of Hindi viseme recognition system. A database has been prepared comprising of ten Hindi sentences uttered by five different speakers. The audio feature based on mel frequency cepstral coefficient (MFCC) has been extracted and subspace-based discrete cosine transform (DCT) was applied to extract visual features. The video-based features were integrated with audio features before using a discriminant function-based classifier for five Hindi viseme classes. Integration of visual features gave an improvement in viseme recognition in case of clean as well as noisy speech. Maximum improvement of 6.67% in accuracy ...

Audio Search Based on Keyword Spotting in Arabic Language

Keyword spotting is an important application of speech recognition. This research introduces a keyword spotting approach to perform audio searching of uttered words in Arabic speech. The matching process depends on the utterance nucleus which is insensitive to its context. For spotting the targeted utterances, the matched nuclei are expanded to cover the whole utterances. Applying this approach to Quran and standard Arabic has promising results. To improve this spotting approach, it is combined with a text search in case of the existence of a transcript. This can be applied on Quran as there is exact correspondence between the audio and text files of each verse. The developed approach starts by text search to identify the verses that include the target utterance(s). For each allocated verse, the occurrence(s) of the target utterance is determined. The targeted utterance (the reference) is manually segmented from an allocated verse. Then Keyword spotting is performed for the extracted reference to the corresponding audio file. The accuracy of the spotted utterances achieved 97%. The experiments showed that the use of the combined text and audio search has reduced the search time by 90% when compared with audio search only tested on the same content. The developed approach has been applied to non transcribed audio files (preaches and News) for searching chosen utterances. The results are promising. The accuracy of spotting was around 84% in case of preaches and 88% in case of the news.

Synote: Collaborative Mobile Learning for All

This paper describes mobile enhancements to Synote, the freely available, award winning, open source, web based application that makes web hosted recordings easier to access, search, manage, and exploit for all learners, teachers and other users. Notes taken live during lectures using Twitter on any mobile device can be automatically uploaded into Synote and synchronised with a recording of the lecture. Syntalk, a mobile speech recognition application enables synchronized live verbal contributions from the class to also be captured on Synote through captions. Synote Mobile has been developed as an accessible cross device and cross browser HTML5 version of Synote. Synote Discussion supports commenting on Synote's Synmark notes stored as discussions in its own database and published as Linked data so they are available for Synote or other systems to use.

SR-NBS: A fast sparse representation based N-best class selector for robust phoneme classification

Although exemplar based approaches have shown good accuracy in classification problems, some limitations are observed in the accuracy of exemplar based automatic speech recognition (ASR) applications. The main limitation of these algorithms is their high computational complexity which makes them difficult to extend to ASR applications. In this paper, an N-best class selector is introduced based on sparse representation (SR) and a tree search strategy. In this approach, the classification is fulfilled in three steps. At first, the set of similar training samples for the specific test sample is selected by k-dimensional (KD) tree search algorithm. Then, an SR based N-best class selector is used to limit the classification among certain classes. This makes the classifier adapt to each test sample and reduces the empirical risk. Finally, a well known low error rate classifier is trained by the selected exemplar samples and the trained classifier is employed to classify among the candidate classes. The algorithm is applied to phoneme classification and it is compared with some well-known phoneme classifiers according to accuracy and complexity issues. By this approach, we obtain competitive classification rate with promising computational complexity in comparison with the state of the art phoneme classifiers in clean and well known acoustic noisy environments which causes this approach become a suitable candidate for ASR applications.

Adaptation of Hidden Markov Models for Recognizing Speech of Reduced Frame Rate

The frame rate of the observation sequence in distributed speech recognition applications may be reduced to suit a resource-limited front-end device. In order to use models trained using full-frame-rate data in the recognition of reduced frame-rate (RFR) data, we propose a method for adapting the transition probabilities of hidden Markov models (HMMs) to match the frame rate of the observation. Experiments on the recognition of clean and noisy connected digits are conducted to evaluate the proposed method. Experimental results show that the proposed method can effectively compensate for the frame-rate mismatch between the training and the test data. Using our adapted model to recognize the RFR speech data, one can significantly reduce the computation time and achieve the same level of accuracy as that of a method, which restores the frame rate using data interpolation.

Low bit rate compression methods of feature vectors for distributed speech recognition

In this paper, we present a family of compression methods based on differential vector quantization (DVQ) for encoding Mel frequency cepstral coefficients (MFCC) in distributed speech recognition (DSR) applications. The proposed techniques benefit from the existence of temporal correlation across consecutive MFCC frames as well as the presence of intra-frame redundancy. We present DVQ schemes based on linear prediction and non-linear methods with multi-layer perceptrons (MLP). In addition to this, we propose the use of a multipath search coding strategy based on the M-algorithm that obtains the sequence of centroids that minimize the quantization error globally instead of selecting the centroids that minimize the quantization error locally in a frame by frame basis. We have evaluated the performance of the proposed methods for two different tasks. On the one hand, two small-size vocabulary databases, Spechdat-Car and Aurora 2, have been considered obtaining negligible degradation in terms of Word Accuracy (around 1%) compared to the unquantized scheme for bit-rates as low as 0.5kbps. On the other hand, for a large vocabulary task (Aurora 4), the proposed method achieves a WER comparable to the unquantized scheme only with 1.6kbps. Moreover, we propose a combined scheme (differential/non-differential) that allows the system to present the same sensitivity to transmission errors than previous multi-frame coding proposals for DSR.

Pipelined language model construction for Polish speech recognition

Abstract The aim of works described in this article is to elaborate and experimentally evaluate a consistent method of Language Model (LM) construction for the sake of Polish speech recognition. In the proposed method we tried to take into account the features and specific problems experienced in practical applications of speech recognition in the Polish language, reach inflection, a loose word order and the tendency for short word deletion. The LM is created in five stages. Each successive stage takes the model prepared at the previous stage and modifies or extends it so as to improve its properties. At the first stage, typical methods of LM smoothing are used to create the initial model. Four most frequently used methods of LM construction are here. At the second stage the model is extended in order to take into account words indirectly co-occurring in the corpus. At the next stage, LM modifications are aimed at reduction of short word deletion errors, which occur frequently in Polish speech recognition. The fourth stage extends the model by insertion of words that were not observed in the corpus. Finally the model is modified so as to assure highly accurate recognition of very important utterances. The performance of the methods applied is tested in four language domains.

Single-Channel Speech Enhancement Techniques for Distant Speech Recognition

AbstractThis article presents an overview of the single-channel dereverberation methods suitable for distant speech recognition (DSR) application. The dereverberation methods are mainly classified based on the domain of enhancement of speech signal captured by a distant microphone. Many single-channel speech enhancement methods focus on either denoising or dereverberating the distorted speech signal. There are very few methods that consider both noise and reverberation effects. Such methods are discussed under a multistage approach in this article. The article concludes with a hypothesis that the methods that do not require an a priori reverberation impulse response is desirable in varying the environmental conditions for DSR applications such as intelligent home and office environments, humanoid robots, and automobiles rather than the methods that require an a priori reverberation impulse response.