Word Accuracy Rate Research Articles

Machine Learning Methods for Automatic Silent Speech Recognition Using a Wearable Graphene Strain Gauge Sensor.

Silent speech recognition is the ability to recognise intended speech without audio information. Useful applications can be found in situations where sound waves are not produced or cannot be heard. Examples include speakers with physical voice impairments or environments in which audio transference is not reliable or secure. Developing a device which can detect non-auditory signals and map them to intended phonation could be used to develop a device to assist in such situations. In this work, we propose a graphene-based strain gauge sensor which can be worn on the throat and detect small muscle movements and vibrations. Machine learning algorithms then decode the non-audio signals and create a prediction on intended speech. The proposed strain gauge sensor is highly wearable, utilising graphene’s unique and beneficial properties including strength, flexibility and high conductivity. A highly flexible and wearable sensor able to pick up small throat movements is fabricated by screen printing graphene onto lycra fabric. A framework for interpreting this information is proposed which explores the use of several machine learning techniques to predict intended words from the signals. A dataset of 15 unique words and four movements, each with 20 repetitions, was developed and used for the training of the machine learning algorithms. The results demonstrate the ability for such sensors to be able to predict spoken words. We produced a word accuracy rate of 55% on the word dataset and 85% on the movements dataset. This work demonstrates a proof-of-concept for the viability of combining a highly wearable graphene strain gauge and machine leaning methods to automate silent speech recognition.

An Effective Conversion of Visemes to Words for High-Performance Automatic Lipreading.

As an alternative approach, viseme-based lipreading systems have demonstrated promising performance results in decoding videos of people uttering entire sentences. However, the overall performance of such systems has been significantly affected by the efficiency of the conversion of visemes to words during the lipreading process. As shown in the literature, the issue has become a bottleneck of such systems where the system’s performance can decrease dramatically from a high classification accuracy of visemes (e.g., over 90%) to a comparatively very low classification accuracy of words (e.g., only just over 60%). The underlying cause of this phenomenon is that roughly half of the words in the English language are homophemes, i.e., a set of visemes can map to multiple words, e.g., “time” and “some”. In this paper, aiming to tackle this issue, a deep learning network model with an Attention based Gated Recurrent Unit is proposed for efficient viseme-to-word conversion and compared against three other approaches. The proposed approach features strong robustness, high efficiency, and short execution time. The approach has been verified with analysis and practical experiments of predicting sentences from benchmark LRS2 and LRS3 datasets. The main contributions of the paper are as follows: (1) A model is developed, which is effective in converting visemes to words, discriminating between homopheme words, and is robust to incorrectly classified visemes; (2) the model proposed uses a few parameters and, therefore, little overhead and time are required to train and execute; and (3) an improved performance in predicting spoken sentences from the LRS2 dataset with an attained word accuracy rate of 79.6%—an improvement of 15.0% compared with the state-of-the-art approaches.

Speech Performance among Healthy Malay Female Speakers during Dual Tasks and Sentence Complexity

Objectives: This study aimed to identify how different dual tasks influence speech performance among young Malay speakers by manipulating the complexity of sentences. Fifteen female speakers (20–30 years old) were audio recorded while executing a series of simple and complex sentence repetition tasks. Methods: Hierarchical linear modeling was conducted to examine the effects of task (speech task only, speech + cognitive tasks, speech + linguistic tasks, and speech + manual tasks) and sentence complexity (simple vs. complex), as well as their interaction on 3 outcome measures (word accuracy rate [WAR], response reaction time [RRT], and duration of sentence [DoS]). Results: Significantly higher WAR were found when participants read the sentences only compared to when they completed another task simultaneously. The pairwise comparison further revealed that the conditions of speech + manual tasks produced the lowest WAR. For RRT, participants responded significantly faster when they only read the sentences (mean = 1.36, SE = 0.17) than when they read the sentences and conducted another task at the same time (mean = 1.66, SE = 0.14; p < 0.0001). A pairwise comparison showed that the conditions of speech + manual tasks produced the longest DoS. Neither the effect of sentence complexity nor the interaction between task and sentence complexity was statistically significant across WAR, RRT, and DoS. Conclusions: The results augment what is known about interference between speech and other concurrent tasks and provides evidence that divided attention affects manual task performance more than other tasks. In addition, sentence complexity did not affect speech performance in healthy adults. This paradigm could be used in future studies to serve as a clinical marker that can potentially distinguish mild cognitive impairment from normal aging.

Development of an on-Premise Indonesian Handwriting Recognition Backend System Using Open Source Deep Learning Solution For Mobile User

Existing handwriting recognition solution on mobile app provides off premise service which means the handwriting is processed in overseas servers. Data sent to abroad servers are not under our control and could be possibly mishandled or misused. As recognizing handwriting is a complex problem, deep learning is needed. This research has the objective of developing an on premise Indonesian handwriting recognition using open source deep learning solution. Comparison of various deep learning solution to be used in the development are done. The deep learning solution will be used to build architectures. Various database format are also compared to decide which format is suitable to gather Indonesian handwriting database. The gathered Indonesian handwriting database and built architectures are used for experiments which consists of number of Convolutional Neural Network (CNN) layers, rotation and noise data augmentation, and Gated Recurrent Unit (GRU) vs Long Short Term Memory (LSTM). Experiment results shows that rotation data augmentation is the parameter to be change to improve word accuracy and Character Error Rate (CER). The improvement is 64.8% and 23.2% to 69.6% and 20.6% respectively.

Evaluation of Intelligent Speech Technology in Epidemic Prevention: Take Iflytek Input Software in Chinese and Japanese Recognition as an Example

This study aims to investigate public’s understanding of intelligent speech technology and to evaluate the word accuracy rate of speech recognition technology in Japanese and Chinese recognition and translation, using the simulating corpus of doctor-patient communication. It is concluded from the questionnaire that currently the public has a high awareness and demand of intelligent speech products. The results of speech recognition experiments show that the word accuracy rates are 99.4% in Mandarin, 96.1% in Japanese, 94.2% in Japanese translation of Chinese, and 82.7% in Chinese translation of Japanese, respectively. In the recognition of Mandarin and Japanese, the errors mainly include missing words and word errors caused by the same or similar pronunciation. In the translation between Japanese and Chinese recognition, the whole sentence omission, sentence patterns, tenses and so on are more common.

Performing predefined tasks using the human–robot interaction on speech recognition for an industrial robot

People who are not experts in robotics can easily implement complex robotic applications by using human–robot interaction (HRI). HRI systems require many complex operations such as robot control, image processing, natural speech recognition, and decision making. In this study, interactive control with an industrial robot was performed by using speech recognition software in the Turkish language. The collected voice data were converted to text data by using automatic speech recognition module based on deep neural networks (DNN). The proposed DNN (p-DNN) was compared to classic classification algorithms. Converted text data was improved in another module to select the process to be applied. According to selected process, position data were defined using image processing. The determined position information was sent to the robot using a fuzzy controller. The developed HRI system was implemented on a KUKA KR Agilus KR6 R900 sixx robot manipulator. The word accuracy rate of the p-DNN model was measured as 90.37%. The developed image processing module and fuzzy controller worked with minimal errors. The contribution of this study is that an industrial robot is easily programming using this software by people who are not experts in robotics and know Turkish.

A modified cascaded neuro-computational model applied to recognition of connected spoken Japanese prefecture words

In this paper, a novel approach of connected spoken word recognition is proposed, based only on a relatively simple artificial neural network model. The model used is a modified version of the previously proposed cascaded neuro-computational model and has a three-layered network structure, where a non-linear metric to each of the second-layer units is newly introduced for performing effectively the pattern matching at the word-feature level. Simulations were conducted using connected speech data sets of a larger lexicon than those used in the previous works; the data sets were comprised of the naturally spoken strings, each string consisting of a varying number of 2–7 words selected from a total of 47 Japanese prefecture names. The simulation results show that the modified model yields the overall recognition performance, i.e., 95.2% in terms of the word accuracy rate, which is comparable to that (98.1%) obtained using a benchmark approach of hidden Markov model with embedded training.

Transcription of Spanish Historical Handwritten Documents with Deep Neural Networks

The digitization of historical handwritten document images is important for the preservation of cultural heritage. Moreover, the transcription of text images obtained from digitization is necessary to provide efficient information access to the content of these documents. Handwritten Text Recognition (HTR) has become an important research topic in the areas of image and computational language processing that allows us to obtain transcriptions from text images. State-of-the-art HTR systems are, however, far from perfect. One difficulty is that they have to cope with image noise and handwriting variability. Another difficulty is the presence of a large amount of Out-Of-Vocabulary (OOV) words in ancient historical texts. A solution to this problem is to use external lexical resources, but such resources might be scarce or unavailable given the nature and the age of such documents. This work proposes a solution to avoid this limitation. It consists of associating a powerful optical recognition system that will cope with image noise and variability, with a language model based on sub-lexical units that will model OOV words. Such a language modeling approach reduces the size of the lexicon while increasing the lexicon coverage. Experiments are first conducted on the publicly available Rodrigo dataset, which contains the digitization of an ancient Spanish manuscript, with a recognizer based on Hidden Markov Models (HMMs). They show that sub-lexical units outperform word units in terms of Word Error Rate (WER), Character Error Rate (CER) and OOV word accuracy rate. This approach is then applied to deep net classifiers, namely Bi-directional Long-Short Term Memory (BLSTMs) and Convolutional Recurrent Neural Nets (CRNNs). Results show that CRNNs outperform HMMs and BLSTMs, reaching the lowest WER and CER for this image dataset and significantly improving OOV recognition.

An experimental framework for Arabic digits speech recognition in noisy environments

In this paper we present an experimental framework for Arabic isolated digits speech recognition named ARADIGITS-2. This framework provides a performance evaluation of Modern Standard Arabic devoted to a Distributed Speech Recognition system, under noisy environments at various Signal-to-Noise Ratio (SNR) levels. The data preparation and the evaluation scripts are designed by deploying a similar methodology to that followed in AURORA-2 database. The original speech data contains a total of 2704 clean utterances, spoken by 112 (56 male and 56 female) Algerian native speakers, down-sampled at 8 kHz. The feature vectors, which consist of a set of Mel Frequency Cepstral Coefficients and log energy, are extracted from speech samples using ETSI Advanced Front-End (ETSI-AFE) standard; whereas, the Hidden Markov Models (HMMs) Toolkit is used for building the speech recognition engine. The recognition task is conducted in speaker-independent mode by considering both word and syllable as acoustic units. Therefore, an optimal fitting of HMM parameters, as well as the temporal derivatives window, is carried out through a series of experiments performed on the two training modes: clean and multi-condition. Better results are obtained by exploiting the polysyllabic nature of Arabic digits. These results show the effectiveness of syllable-like unit in building Arabic digits recognition system, which exceeds word-like unit by an overall Word Accuracy Rate of 0.44 and 0.58% for clean and multi-condition training modes, respectively.

Sign Transition Modeling and a Scalable Solution to Continuous Sign Language Recognition for Real-World Applications

We propose a new approach to modeling transition information between signs in continuous Sign Language Recognition (SLR) and address some scalability issues in designing SLR systems. In contrast to Automatic Speech Recognition (ASR) in which the transition between speech sounds is often brief and mainly addressed by the coarticulation effect, the sign transition in continuous SLR is far from being clear and usually not easily and exactly characterized. Leveraging upon hidden Markov modeling techniques from ASR, we proposed a modeling framework for continuous SLR having the following major advantages, namely: (i) the system is easy to scale up to large-vocabulary SLR; (ii) modeling of signs as well as the transitions between signs is robust even for noisy data collected in real-world SLR; and (iii) extensions to training, decoding, and adaptation are directly applicable even with new deep learning algorithms. A pair of low-cost digital gloves affordable for the deaf and hard of hearing community is used to collect a collection of training and testing data for real-world SLR interaction applications. Evaluated on 1,024 testing sentences from five signers, a word accuracy rate of 87.4% is achieved using a vocabulary of 510 words. The SLR speed is in real time, requiring an average of 0.69s per sentence. The encouraging results indicate that it is feasible to develop real-world SLR applications based on the proposed SLR framework.

Towards Robust Indonesian Speech Recognition with Spontaneous-Speech Adapted Acoustic Models

This paper presents our work in building an Indonesian speech recognizer to handle both spontaneous and dictated speech. The recognizer is based on the Gaussian Mixture and Hidden Markov Models (GMM-HMM). The model is first trained on 73hours of dictated speech and 43.5 minutes of spontaneous speech. The dictated speech is read from prepared transcripts by a diverse group of 244 Indonesian speakers. The spontaneous speech is manually labelled from recordings of an Indonesian parliamentary meeting, and is interspersed with noises and fillers. The resulting triphone model is then adapted only to the spontaneous speech using the Maximum A-posteriori Probability (MAP) method. We evaluate the adapted model using separate dictated and spontaneous evaluation sets. The dictated set consists of speech from 20 speakers totaling 14.5 hours. The spontaneous set is derived from the recording of a regional government meeting, consisting of 1085 utterances totaling 48.5minutes. Evaluation of a MAP-adapted spontaneous set yields a 2.60% absolute increase in Word Accuracy Rate (WAR) over the un-adapted model, outperforming MMI adaptation. Conversely, MMI adaption of the dictated set outperforms the MAP adaptation by achieving an absolute increase of 1.48% in WAR over the un-adapted model. We also demonstrate that fMLLR speaker adaptation is unsuitable for our task due to limited adaptation data.

Intelligibility Assessment and Speech Recognizer Word Accuracy Rate Prediction for Dysarthric Speakers in a Factor Analysis Subspace

Automated intelligibility assessments can support speech and language therapists in determining the type of dysarthria presented by their clients. Such assessments can also help predict how well a person with dysarthria might cope with a voice interface to assistive technology. Our approach to intelligibility assessment is based on iVectors , a set of measures that capture many aspects of a person’s speech, including intelligibility. The major advantage of iVectors is that they compress all acoustic information contained in an utterance into a reduced number of measures, and they are very suitable to be used with simple predictors. We show that intelligibility assessments work best if there is a pre-existing set of words annotated for intelligibility from the speaker to be evaluated, which can be used for training our system. We discuss the implications of our findings for practice.

Automatic assessment of expressive oral reading

We investigated the automatic assessment of expressive children’s oral reading of grade level text passages using a standardized rubric. After a careful review of the reading literature and a close examination of the rubric, we designed a novel set of prosodic and lexical features to characterize fluent expressive reading.A number of complementary sources of information were used to design the features, each of them motivated by research on different components of reading fluency. Features are connected to the child’s reading rate, to the presence and number of pauses, filled-pauses and word-repetitions, the correlation between punctuation marks and pauses, the length of word groupings, syllable stress and duration and the location of pitch peaks and contours.The proposed features were evaluated on a corpus of 783 one-minute reading sessions from 313 students reading grade-leveled passages without assistance (cold unassisted reading). Experimental results show that the proposed lexical and prosodic features provide complementary information and are able to capture the characteristics of expressive reading. The results showed that on both the 2-point and the 4-point expressiveness scales, computer-generated ratings of expressiveness agreed with human raters better than the human raters agreed with each other. The results of the study suggest that automatic assessment of expressive oral reading can be combined with automatic measures of word accuracy and reading rate to produce an accurate multidimensional estimate of children’s oral reading ability.

ASR For Embedded Real Time Applications

The system consists of a standard microprocessor and a hardware accelerator for Gaussian mixture model (GMM) emission probability calculation implemented on a field-programmable gate array. The GMM accelerator is optimized for timing performance by exploiting data parallelism. In order to avoid large memory requirement, the accelerator adopts a double buffering scheme for accessing the acoustic parameters with no assumption made on the access pattern of these parameters. Experiments on widely used benchmark data show that the real-time factor of the proposed system is 0.62, which is about three times faster than the pure software- based baseline system, while the word accuracy rate is preserved at 93.33%. As a part of the recognizer, a new adaptive beam-pruning algorithm is also proposed and implemented, which further reduces the average real- time factor to 0.54 with the word accuracy rate of 93.16%. The proposed speech recognizer is suitable for integration in various types of voice (speech)-controlled applications. Index Terms: Automatic speech recognition (ASR), embedded system, hardware-software codesign, real-time system, softcore-based system.

Hardware–Software Codesign of Automatic Speech Recognition System for Embedded Real-Time Applications

We present a hardware-software coprocessing speech recognizer for real-time embedded applications. The system consists of a standard microprocessor and a hardware accelerator for Gaussian mixture model (GMM) emission probability calculation implemented on a field-programmable gate array. The GMM accelerator is optimized for timing performance by exploiting data parallelism. In order to avoid large memory requirement, the accelerator adopts a double buffering scheme for accessing the acoustic parameters with no assumption made on the access pattern of these parameters. Experiments on widely used benchmark data show that the real-time factor of the proposed system is 0.62, which is about three times faster than the pure software-based baseline system, while the word accuracy rate is preserved at 93.33%. As a part of the recognizer, a new adaptive beam-pruning algorithm is also proposed and implemented, which further reduces the average real-time factor to 0.54 with the word accuracy rate of 93.16%. The proposed speech recognizer is suitable for integration in various types of voice (speech)-controlled applications.

Low Bit-Rate Speech Coding through Quantization of Mel-Frequency Cepstral Coefficients

In this paper, we propose a low bit-rate speech codec based on vector quantization (VQ) of the mel-frequency cepstral coefficients (MFCCs). We begin by showing that if a high-resolution mel-frequency cepstrum (MFC) is computed, good-quality speech reconstruction is possible from the MFCCs despite the lack of phase information. By evaluating the contribution toward speech quality that individual MFCCs make and applying appropriate quantization, our results show that the MFCC-based codec exceeds the state-of-the-art MELPe codec across the entire range of 600-2400 bps, when evaluated with the perceptual evaluation of speech quality (PESQ) (ITU-T recommendation P.862). The main advantage of the proposed codec is in distributed speech recognition (DSR) since the MFCCs can be directly applied thus eliminating additional decode and feature extract stages; furthermore, the proposed codec better preserves the fidelity of MFCCs and better word accuracy rates as compared to CELP and MELPe codecs.

Automated postlaryngectomy telephone test

In this study, an objective version of the postlaryngectomy telephone test (PLTT) for measuring speech intelligibility based on automatic speech recognition is presented. Thirty-one patients with tracheoesophageal substitute voice (25 men and six women, 63.4+/-8.7 years) were evaluated by 11 naïve listeners. The automatic measurement of speech intelligibility was expressed by means of word accuracy and word recognition rates, or the percentage of correctly recognized words from a word sequence. These automatic measures were compared with the subjectively obtained PLTT values. The average PLTT intelligibility of the 11 naïve listeners was 47%; the automatically obtained word accuracy and word recognition rates were much lower (approximately 0% and 15%, respectively). The correlation between subjective and automatic evaluation, however, reached more than 0.9 in some of the examined cases. Automatic speech recognition provides an efficient, objective measure that is equivalent to the overall PLTT intelligibility value.

ASR post-correction for spoken dialogue systems based on semantic, syntactic, lexical and contextual information

This paper proposes a technique to correct speech recognition errors in spoken dialogue systems that presents two main novel contributions. On the one hand, it considers several contexts where a speech recognition result can be corrected. A threshold learnt in the training is used to decide whether the correction must be carried out in the context associated with the current prompt type of a dialogue system, or in another context. On the other hand, the technique deals with the confidence scores of the words employed in the corrections. The correction is carried out at two levels: statistical and linguistic. At the first level the technique employs syntactic–semantic and lexical models, both contextual, to decide whether a recognition result is correct. According to this decision the recognition result may be changed. At the second level the technique employs basic linguistic knowledge to decide about the grammatical correctness of the outcome of the first level. According to this decision the outcome may be changed as well. Experimental results indicate that the technique enhances a dialogue system’s word accuracy, speech understanding, implicit recovery and task completion rates by 8.5%, 16.54%, 4% and 44.17%, respectively.

A feature extraction method using subband based periodicity and aperiodicity decomposition with noise robust frontend processing for automatic speech recognition

This paper proposes a frontend processing technique that employs a speech feature extraction method called Subband based Periodicity and Aperiodicity DEcomposition (SPADE), and examines its validity for automatic speech recognition in noisy environments. SPADE divides speech signals into subband signals, which are then decomposed into their periodic and aperiodic features, and uses both features as speech feature parameters. SPADE employs independent periodicity estimation within each subband and periodicity–aperiodicity decomposition design based on a parallel distributed processing technique motivated by the human speech perception process. Unlike other speech features, this decomposition of speech into two characteristics provides information about periodicities and aperiodicities, and thus allows the utilization of the robustness exhibited by periodic features without losing certain essential information included in aperiodic features. This paper first introduces an implementation of SPADE that operates in the frequency domain, and then examines the validity of combining SPADE with speech enhancement methods. For this examination, we combine SPADE with noise compensation methods that operate in the frequency domain and cepstral normalization methods. In addition, we employ an energy parameter calculation method based on the SPADE framework. An evaluation with the AURORA-2J noisy continuous digit speech recognition database (Japanese AURORA-2) shows that SPADE combined with adaptive Wiener filtering, cepstral normalization, and the energy parameter achieves average word accuracy rates of 82.58% with clean training and 92.55% with multicondition training. These rates are higher than those achieved with ETSI WI008 advanced DSR frontend processing (77.98% and 91.01%, respectively) whose speech feature parameter is based on conventional Mel-frequency cepstral coefficients. By comparison with ETSI WI008 advanced DSR frontend, the proposed method reduces word error rates by 20.9% with clean training and 17.2% with multicondition training. These results confirmed that SPADE combined with noise reduction methods can increase robustness in the presence of noise.

Improving Rapid Unsupervised Speaker Adaptation Based on HMM-Sufficient Statistics in Noisy Environments Using Multi-Template Models

This paper describes the method of using multi-template unsupervised speaker adaptation based on HMM-Sufficient Statistics to push up the adaptation performance while keeping adaptation time within few seconds with just one arbitrary utterance. This adaptation scheme is mainly composed of two processes. The first part is done offline which involves the training of multiple class-dependent acoustic models and the creation of speakers' HMM-Sufficient Statistics based on gender and age. The second part is performed online where adaptation begins using the single utterance of a test speaker. From this utterance, the system will classify the speaker's class and consequently select the N-best neighbor speakers close to the utterance using Gaussian Mixture Models (GMM). The classified speakers' class template model is then adopted as a base model. From this template model, the adapted model is rapidly constructed using the N-best neighbor speakers' HMM-Sufficient Statistics. Experiments in noisy environment conditions with 20 dB, 15 dB and 10 dB SNR office, crowd, booth, and car noise are performed. The proposed multi-template method achieved 89.5% word accuracy rate compared with 88.1% of the conventional single-template method, while the baseline recognition rate without adaptation is 86.4%. Moreover, experiments using Vocal Tract Length Normalization (VTLN) and supervised Maximum Likelihood Linear Regression (MLLR) are also compared.