Real-time Speech Recognition Research Articles

A 3D-printed acoustic triboelectric nanogenerator for quarter-wavelength acoustic energy harvesting and self-powered edge sensing

Abstract Low-frequency acoustic energy harvesting is of great significance in the academic field and industry. In this work, we propose a 3D-printed acoustic triboelectric nanogenerator (A-TENG) with the properties of structural controllability, one-time molding, easy fabrication, and low cost. A quarter-wavelength acoustic resonator system based on the A-TENG is demonstrated and systemically investigated for high-performance acoustic energy scavenging. The system is capable of generating a power output of 4.33 mW under 100 dB sound pressure level excitation. Up to 72 LEDs and a commercial calculator can be directly and continuously driven by the acoustic resonator system, indicating its application as a power source for electronic devices. Furthermore, we develop a self-powered edge sensing system consisting of the A-TENG, an AI speech recognition chip, and control circuits. The speech signals can be firstly converted into electrical signals through the A-TENG, and then recognized and processed by the AI chip with a built-in and pre-trained neural network to control the follow-up circuits. The self-powered edge sensing system is capable of real-time speech recognition without cloud computing, exhibiting great potential in the field of low-power and cost-effective intelligent Internet of Things.

Parallel Implementation of Nios Ii Multiprocessors, Cepstral Coefficients of Mel Frequency and MLP Architecture in Fpga: the Application of Speech Recognition

Speech processing in real time requires the use of fast, reconfigurable electronic circuits capable of handling large amounts of information generated by the audio source. This article presents hardware implementations of a multilayer perceptron (MLP) and the MFCC algorithm for speech recognition. These algorithms have been implemented in hardware and tested in an on-board electronic card based on a reconfigurable circuit (FPGA). We also present a comparative study between several architectures of MLP and with the literature on the level of costs with regard to the surface of silicon, the speed and the computing resources required. Following the FPGA circuit modification, we created NIOSII processors to physically implement the architecture of ANN-type MLPs and MFCC speech recognition algorithms and perform real-time speech recognition functions.

Auditory Measures for the Next Billion Users.

A range of new technologies have the potential to help people, whether traditionally considered hearing impaired or not. These technologies include more sophisticated personal sound amplification products, as well as real-time speech enhancement and speech recognition. They can improve user's communication abilities, but these new approaches require new ways to describe their success and allow engineers to optimize their properties. Speech recognition systems are often optimized using the word-error rate, but when the results are presented in real time, user interface issues become a lot more important than conventional measures of auditory performance. For example, there is a tradeoff between minimizing recognition time (latency) by quickly displaying results versus disturbing the user's cognitive flow by rewriting the results on the screen when the recognizer later needs to change its decisions. This article describes current, new, and future directions for helping billions of people with their hearing. These new technologies bring auditory assistance to new users, especially to those in areas of the world without access to professional medical expertise. In the short term, audio enhancement technologies in inexpensive mobile forms, devices that are quickly becoming necessary to navigate all aspects of our lives, can bring better audio signals to many people. Alternatively, current speech recognition technology may obviate the need for audio amplification or enhancement at all and could be useful for listeners with normal hearing or with hearing loss. With new and dramatically better technology based on deep neural networks, speech enhancement improves the signal to noise ratio, and audio classifiers can recognize sounds in the user's environment. Both use deep neural networks to improve a user's experiences. Longer term, auditory attention decoding is expected to allow our devices to understand where a user is directing their attention and thus allow our devices to respond better to their needs. In all these cases, the technologies turn the hearing assistance problem on its head, and thus require new ways to measure their performance.

Real Time Speech Recognition based on PWP Thresholding and MFCC using SVM

The real-time performance of Automatic Speech Recognition (ASR) is a big challenge and needs high computing capability and exhaustive memory consumption. Getting a robust performance against inevitable various difficult situations such as speaker variations, accents, and noise is a tedious task. It’s crucial to expand new and efficient approaches for speech signal extraction features and pre-processing. In order to fix the high dependency issue related to processing succeeding steps in ARS and enhance the extracted features’ quality, noise robustness can be solved within the ARS extraction block feature, removing implicitly the need for further additional specific compensation parameters or data collection. This paper proposes a new robust acoustic extraction approach development based on a hybrid technique consisting of Perceptual Wavelet Packet (PWP) and Mel Frequency Cepstral Coefficients (MFCCs). The proposed system was implemented on a Rasberry Pi board and its performance was checked in a clean environment, reaching 99% average accuracy. The recognition rate was improved (from 80% to 99%) for the majority of Signal-to-Noise Ratios (SNRs) under real noisy conditions for positive SNRs and considerably improved results especially for negative SNRs.

An 8.93 TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity for On-Device Speech Recognition

Long short-term memory (LSTM) is a type of recurrent neural networks (RNNs), which is widely used for time-series data and speech applications, due to its high accuracy on such tasks. However, LSTMs pose difficulties for efficient hardware implementation because they require a large amount of weight storage and exhibit computation complexity. Prior works have proposed compression techniques to alleviate the storage/computation requirements of LSTMs but elementwise sparsity schemes incur sizable index memory overhead and structured compression techniques report limited compression ratios. In this article, we present an energy-efficient LSTM RNN accelerator, featuring an algorithm-hardware co-optimized memory compression technique called hierarchical coarse-grain sparsity (HCGS). Aided by the HCGS-based blockwise recursive weight compression, we demonstrate LSTM networks with up to 16 $\times $ fewer weights while achieving minimal error rate degradation. The prototype chip fabricated in 65-nm LP CMOS achieves up to 8.93 TOPS/W for real-time speech recognition using compressed LSTMs based on HCGS. HCGS-based LSTMs have demonstrated energy-efficient speech recognition with low error rates for TIMIT, TED-LIUM, and LibriSpeech data sets.

A Lightweight Speaker Recognition System Using Timbre Properties

Speaker recognition is an active research area that contains notable usage in biometric security and authentication system. Currently, there exist many well-performing models in the speaker recognition domain. However, most of the advanced models implement deep learning that requires GPU support for real-time speech recognition, and it is not suitable for low-end devices. In this paper, we propose a lightweight text-independent speaker recognition model based on random forest classifier. It also introduces new features that are used for both speaker verification and identification tasks. The proposed model uses human speech based timbral properties as features that are classified using random forest. Timbre refers to the very basic properties of sound that allow listeners to discriminate among them. The prototype uses seven most actively searched timbre properties, boominess, brightness, depth, hardness, roughness, sharpness, and warmth as features of our speaker recognition model. The experiment is carried out on speaker verification and speaker identification tasks and shows the achievements and drawbacks of the proposed model. In the speaker identification phase, it achieves a maximum accuracy of 78%. On the contrary, in the speaker verification phase, the model maintains an accuracy of 80% having an equal error rate (ERR) of 0.24.

RETRACTED: Real time speech recognition algorithm on embedded system based on continuous Markov model

Abstract Real time speech recognition technology, as a key cross technology in the field of artificial intelligence in recent years, has been widely used in the fields of intelligent voice toys, industrial control and intelligent rehabilitation. Because the real-time speech recognition technology based on embedded technology has obvious advantages in the volume, power consumption and research and development cost of the system, it has become a hot carrier to achieve efficient speech recognition technology. In order to realize a simple and practical real-time speech recognition system based on embedded system, this paper designs a basic framework of machine learning based on Markov random field theory combined with machine learning theory, and studies the algorithm of real-time speech vocabulary matching recognition based on this framework. In detail, the algorithm proposed in this paper will process the speech signal from the aspects of preprocessing, signal detection, feature extraction and quantization. Finally, this paper will build a real-time speech recognition system based on DSP processor. The experimental results show that the real-time speech recognition algorithm proposed in this paper can improve the real-time recognition speed of the system. The corresponding speed changes from about 12 s to about 200 ms, and the corresponding real-time accuracy rate increases to about 95%.

The Pransky interview: Dr James Kuffner, CEO at Toyota Research Institute Advanced Development, Coinventor of the rapidly, exploring random tree algorithm

PurposeThe following article is a “Q&A interview” conducted by Joanne Pransky of Industrial Robot Journal as a method to impart the combined technological, business and personal experience of a prominent, robotic industry PhD-turned entrepreneur regarding his pioneering efforts of bringing technological inventions to market. The paper aims to discuss these issues.Design/methodology/approachThe interviewee is Dr James Kuffner, CEO at Toyota Research Institute Advanced Development (TRI-AD). Kuffner is a proven entrepreneur and inventor in robot and motion planning and cloud robotics. In this interview, Kuffner shares his personal and professional journey from conceptualization to commercial realization.FindingsDr Kuffner received BS, MS and PhD degrees from the Stanford University’s Department of Computer Science Robotics Laboratory. He was a Japan Society for the Promotion of Science (JSPS) Postdoctoral Research Fellow at the University of Tokyo where he worked on software and planning algorithms for humanoid robots. He joined the faculty at Carnegie Mellon University’s Robotics Institute in 2002 where he served until March 2018. Kuffner was a Research Scientist and Engineering Director at Google from 2009 to 2016. In January 2016, he joined TRI where he was appointed the Chief Technology Officer and Area Lead, Cloud Intelligence and is presently an Executive Advisor. He has been CEO of TRI-AD since April of 2018.Originality/valueDr Kuffner is perhaps best known as the co-inventor of the rapidly exploring random tree (RRT) algorithm, which has become a key standard benchmark for robot motion planning. He is also known for introducing the term “Cloud Robotics” in 2010 to describe how network-connected robots could take advantage of distributed computation and data stored in the cloud. Kuffner was part of the initial engineering team that built Google’s self-driving car. He was appointed Head of Google’s Robotics Division in 2014, which he co-founded with Andy Rubin to help realize the original Cloud Robotics concept. Kuffner also co-founded Motion Factory, where he was the Senior Software Engineer and a member of the engineering team to develop C++ based authoring tools for high-level graphic animation and interactive multimedia content. Motion Factory was acquired by SoftImage in 2000. In May 2007, Kuffner founded, and became the Director of Robot Autonomy where he coordinated research and software consulting for industrial and consumer robotics applications. In 2008, he assisted in the iOS development of Jibbigo, the first on-phone, real-time speech recognition, translation and speech synthesis application for the iPhone. Jibbigo was acquired by Facebook in 2013. Kuffner is one of the most highly cited authors in the field of robotics and motion planning, with over 15,000 citations. He has published over 125 technical papers and was issued more than 50 patents related to robotics and computer vision technology.

Prehospital Cerebrovascular Accident Detection using Artificial Intelligence Powered Mobile Devices

Abstract Cerebrovascular Accident (CVA) is the second leading cause of death in the world while also being the plurality cause of disability in adults. A definitive factor for survivability and successful recovery of a patient is the time passage from the onset of symptoms to the administration of medical treatment. This paper introduces Stroke Help, a mobile application utilizing various mobile technologies together with Artificial Intelligence algorithms in order to quickly detect CVA in either the user or someone the user is concerned about. The application implements the well known F.A.S.T. test making use of real-time face detection, speech recognition, and other artificial intelligence techniques applied over common sensors found in modern mobile phones. In addition of detecting whether there is a high probability a patient is suffering from a stroke, the application will calculate an approximated Japan Urgent Stroke Triage (JUST) score utilized in identifying the specific type of stroke, very important information for medical staff to potentially reduce the time required to evaluate the patient before beginning the appropriate treatment. We will also present additional crucial functionalities such as notifying contacts, identifying the closest clinics capable of treating CVA, making our solution a complete approach.

Online Speaker Adaptation Using Memory-Aware Networks for Speech Recognition

In our previous work, we introduced our attention-based speaker adaptation method, which has been proved to be an efficient online speaker adaptation method for real-time speech recognition. In this paper, we present a more complete framework of this method named memory-aware networks, which consists of the main network, the memory module, the attention module and the connection module. A gate mechanism and a multiple-connections strategy are presented to connect the memory with the main network in order to take full advantage of the memory. An auxiliary speaker classification task is provided to improve the accuracy of the attention module. The fixed-size ordinally forgetting encoding method is used together with average pooling to gather both short-term and long-term information. Furthermore, instead of only using traditional speaker embeddings such as i-vectors or d-vectors as the memory, we design a new form of memory called residual vectors, which can represent different pronunciation habits. Experiments on both the Switchboard and AISHELL-2 tasks show that our method can perform online speaker adaptation very well with no additional adaptation data and with only a relative 3% increase in decoding computation complexity. Under the cross-entropy criterion, our method achieves a relative word error rate reduction of 9.4% and 8.3% compared to that of the speaker-independent model on the Switchboard task and the AISHELL-2 task, respectively, and approximately 7.0% compared to that of the traditional d-vector-based speaker adaptation method.

An Ultra-Low Power Binarized Convolutional Neural Network-Based Speech Recognition Processor With On-Chip Self-Learning

Always-on speech interfaces are prevailing in human-machine interaction, especially on wearable devices, Internet of Things, etc., which benefits from the recent breakthroughs in deep learning. For battery-powered devices, ultra-low power and real-time processing are critical. However, the massive memory access and computation of deep neural networks (DNNs) lead to long latency and huge energy consumption, which hinder their further integration in battery-powered devices. Extremely low-bit quantization shows the potential to enhance energy efficiency and speed by orders of magnitudes, but it suffers from the degradation of recognition accuracy. In this paper, we propose a binarized convolutional neural network (BCNN) based speech recognition processor, integrated with on-chip self-learning mechanism to compensate the accuracy loss caused by low-precision. We optimize the BCNN architecture by eliminating the computation redundancy, compressing the weights and tailoring approximate circuits. Fabricated in 28 nm CMOS, this processor supports real time speech recognition with power consumption of $141~\mu \text{W}$ and energy efficiency of 2.46 pJ/Neuron. Compared with state-of-the-art speech recognition implementations, this processor achieves $2.5\times $ reduction on energy consumption per neuron, and $8.0\times $ energy reduction per speech frame.

A Systematic Review of Health Dialog Systems.

Health dialog systems have seen increased adoption by patients, hospitals, and universities due to the confluence of advancements in machine learning and the ubiquity of high-performance hardware that supports real-time speech recognition, high-fidelity text-to-speech, and semantic understanding of natural language. This review seeks to enumerate opportunities to apply dialog systems toward the improvement of health outcomes while identifying both gaps in the current literature that may impede their implementation and recommendations that may improve their success in medical practice. A search over PubMed and the ACM Digital Library was conducted on September 12, 2017 to collect all articles related to dialog systems within the domain of health care. These results were screened for eligibility with the main criteria being a peer-reviewed study of a system that includes both a natural language interface and either end-user testing or practical implementation. Forty-six studies met the inclusion criteria including 24 quasi-experimental studies, 16 randomized control trials, 2 case-control studies, 2 prospective cohort studies, 1 system description, and 1 human-computer conversation analysis. These studies evaluated dialog systems in five application domains: medical education (n = 20), clinical processes (n = 14), mental health (n = 5), personal health agents (n = 5), and patient education (n = 2). We found that dialog systems have been widely applied to health care; however, most studies are not reproducible making direct comparison between systems and independent confirmation of findings difficult. Widespread adoption will also require the adoption of standard evaluation and reporting methods for health dialog systems to demonstrate clinical significance.

Real-Time Indonesian Language Speech Recognition with MFCC Algorithms and Python-Based SVM

Abstract — Automatic Speech Recognition (ASR) is a technology that uses machines to process and recognize human voice. One way to increase recognition rate is to use a model of language you want to recognize. In this paper, a speech recognition application is introduced to recognize words "atas" (up), "bawah" (down), "kanan" (right), and "kiri" (left). This research used 400 samples of speech data, 75 samples from each word for training data and 25 samples for each word for test data. This speech recognition system was designed using Mel Frequency Cepstral Coefficient (MFCC) as many as 13 coefficients as features and Support Vector Machine (SVM) as identifiers. The system was tested with linear kernels and RBF, various cost values, and three sample sizes (n = 25, 75, 50). The best average accuracy value was obtained from SVM using linear kernels, a cost value of 100 and a data set consisted of 75 samples from each class. During the training phase, the system showed a f1-score (trade-off value between precision and recall) of 80% for the word "atas", 86% for the word "bawah", 81% for the word "kanan", and 100% for the word "kiri". Whereas by using 25 new samples per class for system testing phase, the f1-score was 76% for the "atas" class, 54% for the "bawah" class, 44% for the "kanan" class, and 100% for the "kiri" class.

AI-Josyu: Thinking Support System in Class by Real-time Speech Recognition and Keyword Extraction

In this paper, we present a thinking support system, AI-Josyu. This system also operates as a class support system which helps to teachers for lightening their work. AI-Josyu is implemented based on media-driven real-time content management framework. The system links real world media and legacy media contents together. In resent years, it is easier to collect a large amount of various kinds of data which are created with sensors in the real world. The system realizes interconnection and utilization of legacy media contents. The legacy media contents are generated and scattered on the Internet. The framework has four modules, which are called “acquisition,” “extraction,” “selection,” and “retrieval.” The real world media and the legacy media contents are interconnected by these modules. This interconnection includes semantic components. This system records teacher's voice of its lecture in real time and presents retrieved legacy media contents corresponding to subject of the lecture. By this presentation, preparing of the legacy contents is not required. This system automatically retrieves and shows the legacy media contents. This system helps students to understand contents of the lecture. In addition, the system attends to expansion of ideas. We constructed the system and conducted the demonstration in class. It shows that the system is helpful to teacher and students for expansion of thinking.

Real-time transcription, keyword spotting, archival and retrieval for telugu TV news using ASR

The main objective of this paper is to describe the system developed for transcription, keyword spotting and alerting, archival, and retrieval for broadcasted Telugu TV news. Real-time automatic speech recognition system is developed for searching the given keywords from the transcribed text of the broadcasted Telugu TV news. The user can set the required keywords and the system will continuously track the audio and alert in real time the user as and when the keywords appeared in the audio. The system will transcribe and index every word with time. As and when a user wanted to retrieve the data with a particular word, it will search the transcribed text in the archives and play the corresponding audio, video along with scrolling of transcribed text of audio. The system is developed fully with GUI for giving the keywords either through microphone or keyboard by transliteration in Telugu. The ASR system is developed using the annotated speech corpus of 65 h duration with two hundred thousand unique words. The acoustic models are developed using subspace Gaussian mixture model. The language modeling is developed with Witten Bell and Kneser–Ney smoothing techniques

A Robust Feature Extraction Method for Real-Time Speech Recognition System on a Raspberry Pi 3 Board

The development of a real-time automatic speech recognition system (ASR) better adapted to environmental variabilities, such as noisy surroundings, speaker variations and accents has become a high priority. Robustness is required, and it can be performed at the feature extraction stage which avoids the need for other pre-processing steps. In this paper, a new robust feature extraction method for real-time ASR system is presented. A combination of Mel-frequency cepstral coefficients (MFCC) and discrete wavelet transform (DWT) is proposed. This hybrid system can conserve more extracted speech features which tend to be invariant to noise. The main idea is to extract MFCC features by denoising the obtained coefficients in the wavelet domain by using a median filter (MF). The proposed system has been implemented on Raspberry Pi 3 which is a suitable platform for real-time requirements. The experiments showed a high recognition rate (100%) in clean environment and satisfying results (ranging from 80% to 100%) in noisy environments at different signal to noise ratios (SNRs).

Offline to online speaker adaptation for real-time deep neural network based LVCSR systems

In this study, we investigate an offline to online strategy for speaker adaptation of automatic speech recognition systems. These systems are trained using the traditional feed-forward and the recent proposed lattice-free maximum mutual information (MMI) time-delay deep neural networks. In this strategy, the test speaker identity is modeled as an iVector which is offline estimated and then used in an online style during speech decoding. In order to ensure the quality of iVectors, we introduce a speaker enrollment stage which can ensure sufficient reliable speech for estimating an accurate and stable offline iVector. Furthermore, different iVector estimation techniques are also reviewed and investigated for speaker adaptation in large vocabulary continuous speech recognition (LVCSR) tasks. Experimental results on several real-time speech recognition tasks demonstrate that, the proposed strategy can not only provide a fast decoding speed, but also can result in significant reductions in word error rates (WERs) than traditional iVector based speaker adaptation frameworks.

A probabilistic feature based SVM model for Hindi/English speech recognition

Real time speech recognition has various challenges including noise, turbulence, language and crosstalk problem. In this paper, multi-phase hybridization is applied to cover these challenges and to provide effective speech recognition. The model is explicitly divided into three main stages where each stage is implicitly divided into several sub-stages to provide specific problem solution. The proposed hybrid model resolved the problem of acoustic turbulence, background noise and instrumentation noise problem at the earlier stage. The rectified speech signals are processed using ICA and Fuzzy-HMM approach to generate the structural and statistical features. In this stage, the signal is divided in smaller linear blocks to extract the features. Later on, fuzzy-weighted SVM is implied to recognize the speech signal. The experimentation is applied on Hindi and English characters and sentence datasets. The comparative results are derived against BPNN and PCA models for different sample sets. The comparative results obtained from model signifies that the model has improved the recognition rate effectively.

Real-time classification of auditory sentences using evoked cortical activity in humans

Objective. Recent research has characterized the anatomical and functional basis of speech perception in the human auditory cortex. These advances have made it possible to decode speech information from activity in brain regions like the superior temporal gyrus, but no published work has demonstrated this ability in real-time, which is necessary for neuroprosthetic brain–computer interfaces. Approach. Here, we introduce a real-time neural speech recognition (rtNSR) software package, which was used to classify spoken input from high-resolution electrocorticography signals in real-time. We tested the system with two human subjects implanted with electrode arrays over the lateral brain surface. Subjects listened to multiple repetitions of ten sentences, and rtNSR classified what was heard in real-time from neural activity patterns using direct sentence-level and HMM-based phoneme-level classification schemes. Main results. We observed single-trial sentence classification accuracies of or higher for each subject with less than 7 minutes of training data, demonstrating the ability of rtNSR to use cortical recordings to perform accurate real-time speech decoding in a limited vocabulary setting. Significance. Further development and testing of the package with different speech paradigms could influence the design of future speech neuroprosthetic applications.

, ,

There have being great efforts made in the development of automated Instrumentation system for speech recognition (AISR) to provide a two-way communication between deaf and vocal people. This system performance achievable with the output of current real-time speech recognition systems would be extremely poor relative to normal speech reception. An alternate application of AISR technology to aid the hearing impaired would derive cues from the acoustical speech signal that could be used to supplement speechreading. We propose a study of highly trained receivers of speech signal that indicates that nearly perfect reception of everyday connected speech materials can be achieved at near normal speaking rates. To understand the accuracy that might be achieved with automatically generated cue symbols for visual representation. The system uses (HMM) for recognition of voiced data & Euclidian distance approach for sign language. The proposed task is a complementary work to the ongoing research work for recognizing the finger movement of a vocally disabled person to speech signal called. A New communication Paradigm: “Action-To-Speech”