Speech Commands Research Articles

Transfer of Learning: What Does It Tell Us About Speech Production Units?

Words, syllables, and phonemes have each been regarded as basic encoding units of speech production in various psycholinguistic models. The present article investigates the role of each unit in the interface with speech articulation, using a paradigm from motor control research. Seventy-six native speakers of French were trained to change their production of /be/ in response to an auditory feedback perturbation (auditory-motor learning). We then assessed the magnitude of learning transfer from /be/ to the syllables in 2 pseudowords (/bepe/ and /pebe/) and 1 real word (/bebe/) as well as the aftereffect on the same utterance (/be/) with a between-subjects design. This made it possible to contrast the amplitude of transfer at the levels of the utterance, the syllable, and the phoneme, depending on the position in the word. Linear mixed models allowed us to study the amplitude as well as the dynamics of the transfer and the aftereffect over trials. Transfer from the training utterance /be/ was observed for all vowels of the test utterances but was larger to the syllable /be/ than to the syllable /pe/ at word-initial position and larger to the 1st syllable than to the 2nd syllable in the utterance. Our study suggests that words, syllables, and phonemes may all contribute to the definition of speech motor commands. In addition, the observation of a serial order effect raises new questions related to the connection between psycholinguistic models and speech motor control approaches.

Integrated Speaker and Speech Recognition for Wheel Chair Movement Using Artificial Intelligence

A speech signal is a result of the constrictions of vocal tract and different sounds can be generated by different vocal tract constrictions. A speech signal carries two things i.e. speaker's identity and meaning. For a specific applications of speaker and speech recognition like voice operated wheel chair, both the speaker and speech is to be recognized for movement of wheel chair. Automation in wheelchair is, today's requirement as the numbers of people are increasing with disabilities like injuries in spine, amputation, impairments in hands etc. They need assistance for moving their wheel chair. Voice operated wheel chair is one of the solution. The intention of this study is to use a speaker and speech dependent system to control the wheelchair and minimize the risk of unwanted accident. We have proposed a system in which speaker (patient) as well as speech (commands) is recognized based upon the acoustic features like Mel Frequency Cepstral Coefficients (MFCC). Optimization of the features using Artificial Bee Algorithm (ABC) is done to gain good accuracy with artificial intelligence technique as a classifier. We have tested our system on standard dataset (TIDIGITS) and our own prepared dataset. Also, validation of proposed work is done by generating control signal to actuate the wheel chair in real time scenario.

Listening to the World Improves Speech Command Recognition

We study transfer learning in convolutional network architectures applied to the task of recognizing audio, such as environmental sound events and speech commands. Our key finding is that not only is it possible to transfer representations from an unrelated task like environmental sound classification to a voice-focused task like speech command recognition, but also that doing so improves accuracies significantly. We also investigate the effect of increased model capacity for transfer learning audio, by first validating known results from the field of Computer Vision of achieving better accuracies with increasingly deeper networks on two audio datasets: UrbanSound8k and Google Speech Commands. Then we propose a simple multiscale input representation using dilated convolutions and show that it is able to aggregate larger contexts and increase classification performance. Further, the models trained using a combination of transfer learning and multiscale input representations need only 50% of the training data to achieve similar accuracies as a freshly trained model with 100% of the training data. Finally, we demonstrate a positive interaction effect for the multiscale input and transfer learning, making a case for the joint application of the two techniques.

Performance of WUW and general ASR speech recognition systems in different acoustic environments

The Automatic Speech Recognition (ARS) Systems have become popular in the different areas of applications, such as education assistance, medical assistance, personal assistant, robotics and vehicles, telecommunications Industry, disabilities systems, home automation, and security access control. All these applications in life will increase the number of speech commands, which are Wake-up-words. The WUW speech recognition is similar to Key-Word spotting. However, in this paper, we propose an approach that will be used to detect the Wake-up-word (WUW) as a command with 100% accuracy and to provide a whole speech recognition system that can work in both WUW and General ASR systems with high accuracy. Also, this new ASR system will be used to solve one of the biggest problems in speech recognition, which is how to discriminate between the uses of a word or phrase in an alerting and a referential context. For example, in the use of the word “Computer'' in an alerting context is Computer, show me the chart? '' and in a referential context it is “Every computer should have a speaker. Moreover, we tested the new system with different acoustic environments, such as different background noise levels, different speaker distances to the microphone, and different speakers.

Pengenalan dan Analisis Ucapan pada Sistem Kontrol Perangkat Listrik Menggunakan Arduino Uno

This study aims to develop an electrical device control system using speech recognition and perform the analysis factors that affect the accuracy of speech recognition. The system used the Arduino Uno as the main control board of the system, an Elechouse's voice recognition module as speech processing device and several output devices such as relays, LEDs and an LCD. This research resulted in 92,7% accuracy of speech recognition at the ideal condition and 66,85% at the noise condition. This study shows that the frequency, amplitude, slow tempo and timbre of command speech greatly affect the success of speech recognition.

Towards Robust Human-Robot Collaborative Manufacturing: Multimodal Fusion

Intuitive and robust multimodal robot control is the key toward human–robot collaboration (HRC) for manufacturing systems. Multimodal robot control methods were introduced in previous studies. The methods allow human operators to control robot intuitively without programming brand-specific code. However, most of the multimodal robot control methods are unreliable because the feature representations are not shared across multiple modalities. To target this problem, a deep learning-based multimodal fusion architecture is proposed in this paper for robust multimodal HRC manufacturing systems. The proposed architecture consists of three modalities: speech command, hand motion, and body motion. Three unimodal models are first trained to extract features, which are further fused for representation sharing. Experiments show that the proposed multimodal fusion model outperforms the three unimodal models. This paper indicates a great potential to apply the proposed multimodal fusion architecture to robust HRC manufacturing systems.

Real-Time Control of Mobile Robot Using HMM-based Speech Recognition System

Human-robot interaction (HRI) is a significant area of interest in robotics which has attracted a wide variety of studies in recent years. In order to provide natural human-robot interaction, robots will have to acquire the skills to detect and to integrate meaningfully information from multiple modalities. In this paper, a practical speech-controlled mobile robot car system is presented and discussed. In this study the developed Hidden Markov Model (HMM) with separate word recognition system and real-time control were obtained on a mobile robot. Mel-Frequency Cepstral Coefficients (MFCC) were applied as features for the control design of mobile robot. In the study, 270 speech commands (İLERİ=forward, GERİ=backward, DUR=stop, SAĞA=right, SOLA=left) which are collected from 54 different people were applied to a series of mathematical operations and 12 cepstral coefficients were derived. Therefore, a database was generated by 12 cepstral coefficients. Thus, HMM model was trained and tested according to database. Speech data were classified in two groups as 90% training data and 10% test data. The recognition success rate of test commands was measured 94%.

Development of Gesture-based Commands for Natural User Interfaces

Human-computer interfaces are changing to meet the evolving needs of users and overcome limitations of previous generations of computer systems. The current state of computers consists largely of graphical user interfaces (GUI) that incorporate windows, icons, menus, and pointers (WIMPs) as visual representations of computer interactions controlled via user input on a mouse and keyboard. Although this model of interface has dominated human-computer interaction for decades, WIMPs require an extra step between the user’s intent and the computer action, imposing both limitations on the interaction and introducing cognitive demands (van Dam, 1997). Alternatively, natural user interfaces (NUI) employ input methods such as speech, touch, and gesture commands. With NUIs, users can interact directly with the computer without using an intermediary device (e.g., mouse, keyboard). Using the body as an input device may be more “natural” because it allows the user to apply existing knowledge of how to interact with the world (Roupé, Bosch-Sijtsema, & Johansson, 2014). To utilize the potential of natural interfaces, research must first determine what interactions can be considered natural. For the purpose of this paper, we focus on the naturalness of gesture-based interfaces. The purpose of this study was to determine how people perform natural gesture-based computer actions. To answer this question, we first narrowed down potential gestures that would be considered natural for an action. In a previous study, participants ( n=17) were asked how they would gesture to interact with a computer to complete a series of actions. After narrowing down the potential natural gestures by calculating the most frequently performed gestures for each action, we asked participants ( n=188) to rate the naturalness of the gestures in the current study. Participants each watched 26 videos of gestures (3-5 seconds each) and were asked how natural or arbitrary they interpreted each gesture for the series of computer commands (e.g., move object left, shrink object, select object, etc.). The gestures in these videos included the 17 gestures that were most often performed in the previous study in which participants were asked what gesture they would naturally use to complete the computer actions. Nine gestures were also included that were created arbitrarily to act as a comparison to the natural gestures. By analyzing the ratings on a continuum from “Completely Arbitrary” to “Completely Natural,” we found that the natural gestures people produced in the first study were also interpreted as the intended action by this separate sample of participants. All the gestures that were rated as either “Mostly Natural” or “Completely Natural” by participants corresponded to how the object manipulation would be performed physically. For example, the gesture video that depicts a fist closing was rated as “natural” by participants for the action of “selecting an object.” All of the gestures that were created arbitrarily were interpreted as “arbitrary” when they did not correspond to the physical action. Determining how people naturally gesture computer commands and how people interpret those gestures is useful because it can inform the development of NUIs and contributes to the literature on what makes gestures seem “natural.”

Sensory attenuation of self-initiated sounds maps onto habitual associations between motor action and sound.

Sensory attenuation refers to reduced brain responses to self-initiated sensations relative to those produced by the external world, a distinction that is vital for dynamic motor control and our sense of agency. Typically, willed vocalizations elicit larger N1 reduction of the auditory evoked potential compared to indirectly evoked sounds, such as tones generated by button-presses, which is attributed to the prediction and cancellation of incoming signals enabled by speech motor commands. However, physical confounds exist, including different stimuli and the increased motor artefact associated with mouth vs. finger movements. The present study investigated N1 attenuation to physically identical sounds evoked by hand, eye, and mouth-initiated movements. Twenty-eight healthy participants had their electroencephalogram (EEG) recorded as they blew into a microphone, pressed a button, or moved their eye to generate a pure tone. We found that N1 and P2 response was most reduced in the blow initiation condition, and that both blow and button-press but not saccade initiated tones elicited significantly reduced N1 and P2 amplitude compared to external initiation. This indicates that the eliciting motor action markedly influences ERP response to auditory stimuli. Given that saccades are never associated with sounds, finger movements sometimes are, and mouth movements often are, the pattern of results suggests that N1 attenuation to self-initiated sounds may depend on existing associations between the initiating action and resultant sensation.

Voice controlled Smart Electric-Powered wheelchair based on Artificial Neural Network

Electric Powered Wheelchair (EPW) used by people with disabilities, who cannot move or with a limited ability to move. The traditional technique that used to control the powered wheelchair can be a challenging and stressful operation. Therefore, there is a demand to find alternative ways to perform this process. In that, the voice recognition is a natural form of pointing that can use to replace the joystick whilst still allowing for similar control. Neural network one of the effective ways to recognize the speech commands, because of its ability to grant a trainable powered wheelchair controller with a convenient way, ease of use and attractiveness. In this situation, the neural networks are providing a trainable system for each person irrespectively of his disability. Smart Electric-Powered Wheelchair (SEPW) is an electric-powered wheelchair based on Artificial Neural Networks (ANN) Technique can recognize the voice commands and replace the traditional control of powered wheelchairs with intelligent technology. So, a neural network for the feed-forward Multi-layer perceptron has been trained to recognize isolated spoken words (commands), such as ‘right, ‘left,' ‘stop' and so on. Initially, the design of the system's network started with one hidden layer using five neurons (according to [1] formula). During the training process, the number of hidden neurons has been increased to keep the mean square error (MSE) of the system minimum as possible. Furthermore, the system structure is designed based on a Multi Layers Perceptron (MLP) neural system for five output neurons, which represent the powered wheelchair's commands

Integrating iRobot-Arduino-Kinect Systems: An Educational Voice and Keyboard Operated Wireless Mobile Robot

This paper presents use of iRobot Create Programmable Robot for real time wireless controlling via speech and keyboard commands in a domestic environment. Commercial hardware such as iRobot create 2 programmable robot, Kinect V2, Arduino Uno and NRF24L01 radio frequency module were used, based on open-source code in the implementation. At the receiver end iRobot programmable robot was used as the mobile robot. At the transmission end, a PC based system was used. The keyboard controlling system is designed to control the directions of the moving robot. In the voice controlling unit, voice commands were recognized using the inbuilt speech recognition engine of the Kinect sensor. We provide comprehensive analysis of the performance of the developed robot system in an indoor environment. The results show that 89% of accuracy for untrained non-native (Sri Lankan) English speakers in British accent.

Working memory facilitates the detection and correction of feedback errors in vocal pitch regulation

In speech processing, information related to the speech motor command and sensory re-afference can be stored in working memory (WM) within a feedback circuit and recalled when needed to adjust the motor activity. Whether WM facilitates the online monitoring of speech motor control, however, remains unclear. The present event-related potential study sought to examine the impact of WM on the auditory-motor processing of pitch feedback errors. Participants sustained a vowel phonation while hearing their voice pitch-shifted + 200 or + 500 cents five times. In the WM task, participants were asked to determine whether the sequence of 5 pitch shifts was different or not between two consecutive vocalizations. In the control task, they did nothing but maintaining their vocalizations steady when exposed to pitch-shifted auditory feedback. The behavioral results revealed a significant increase of vocal responses in the WM task as compared to the control task. At the cortical level, the WM task elicited significantly larger N1 responses and smaller P2 responses than the control task. Taken together, these findings demonstrate the influence of WM on the neurobehavioral responses to pitch-shifted voice auditory feedback, indicating that WM can facilitate the detection and correction of pitch feedback errors in vocal motor control.

FPGA-Based Robust Wireless Speech Motion Control for Home Service Robot Subject to Environmental Noises

In this paper, a robust speech recognition system for the recognition of the speeches subjected to environmental noise is designed and implemented on FPGA to control a home service robot wirelessly. An empirical mode decomposition is used to separate the clean speeches from the speech signals contaminated by environmental noise. To improve the recognition speed, instead of continuous hidden Markov model (CHMM), Discrete HMM (DHMM) is used here to reduce the computation load during speech recognition. However, to compensate the decreased speech recognition rate using DHMM, this paper uses fuzzy vector quantization (FVQ) on the modeling of DHMM to improve the speech recognition rates. It will be shown that the computation time just increases a little, while the speech recognition rates increase much when the FVQ is applied. Finally, combining a wireless module, a FPGA-based speech recognition system is designed to control the motions of a home service robot wirelessly via speech commands under some environmental noises. The performance of the designed system will be demonstrated in the end of this paper.

APPROACH TO CREATING SETS OF REFERENCE SPEECH COMMANDS USING ONTOLOGY

This article describes the method of formation of speech commands reference library using ontology to increase their further recognition. The article presents the formal model of a subject OWL-ontology, used as a base of knowledge in the process of formation and expansion of the library of reference teams in the process of recognition of voice commands. An algorithm for retrieval of the list of synonyms of input speech commands, based on the subject of OWLontology is also described. The article describes the research methodology, as well as the results of experiments on the formation of variants of the reference library and selection of a library with the most recognizable commands using OWL-ontology of civil aircraft voice command control. In conclusion, the results of experiments demonstrating the effectiveness of this approach are presented.

Design of Voice Controlled Smart Wheelchair

This paper describes the design of a smart, motorized, voice controlled wheelchair using embedded system. Proposed design supports voice activation system for physically disabled persons incorporating manual operation. Arduino microcontroller and speaker dependent voice recognition processor have been used to support the navigation of the wheel chair. The direction and velocity of the chair are controlled by pre-defined Arabic voice commands. The speaker dependent, isolated word recognition system (IWRS) for a definite utterance of Arabic words to suit the patient's requirements has been programmed and successfully demonstrated. The technique of speech signal processing for extraction of sound parameters, noise removal, intensity and normalization of time , and features matching etc. have been done with the speech processor HM2007that being embedded efficiently in real time. Arduino receives the coded digital signals from the IWRS which being properly recognizes voice commands in order to control the function of the chair accordingly. The wheelchair does not respond to a false speech command. The overall mechanical assembly is driven by two14A/24V/200Watt DC motors having an engagement/disengagement clutch and speed reduction gear with built-in locking control. The system is tested using a speech password to start operation and seven Arabic commands to control motion: Amam (forward), Saree'(fast), Batee' (slow), Khalf (backward), Yameen (right), Yesar (left), Tawaqaf (stop). It proved a good working in clear and noisy environments with safe movement.

Attentional demands modulate sensorimotor learning induced by persistent exposure to changes in auditory feedback.

Hearing one's own voice is important for regulating ongoing speech and for mapping speech sounds onto articulator movements. However, it is currently unknown whether attention mediates changes in the relationship between motor commands and their acoustic output, which are necessary as growth and aging inevitably cause changes to the vocal tract. In this study, participants produced vocalizations while they heard their vocal pitch persistently shifted downward one semitone in both single- and dual-task conditions. During the single-task condition, participants vocalized while passively viewing a visual stream. During the dual-task condition, participants vocalized while also monitoring a visual stream for target letters, forcing participants to divide their attention. Participants' vocal pitch was measured across each vocalization, to index the extent to which their ongoing vocalization was modified as a result of the deviant auditory feedback. Smaller compensatory responses were recorded during the dual-task condition, suggesting that divided attention interfered with the use of auditory feedback for the regulation of ongoing vocalizations. Participants' vocal pitch was also measured at the beginning of each vocalization, before auditory feedback was available, to assess the extent to which the deviant auditory feedback was used to modify subsequent speech motor commands. Smaller changes in vocal pitch at vocalization onset were recorded during the dual-task condition, suggesting that divided attention diminished sensorimotor learning. Together, the results of this study suggest that attention is required for the speech motor control system to make optimal use of auditory feedback for the regulation and planning of speech motor commands.

Sensorimotor learning in children and adults: Exposure to frequency-altered auditory feedback during speech production

Auditory feedback plays an important role in the acquisition of fluent speech; however, this role may change once speech is acquired and individuals no longer experience persistent developmental changes to the brain and vocal tract. For this reason, we investigated whether the role of auditory feedback in sensorimotor learning differs across children and adult speakers. Participants produced vocalizations while they heard their vocal pitch predictably or unpredictably shifted downward one semitone. The participants’ vocal pitches were measured at the beginning of each vocalization, before auditory feedback was available, to assess the extent to which the deviant auditory feedback modified subsequent speech motor commands. Sensorimotor learning was observed in both children and adults, with participants’ initial vocal pitch increasing following trials where they were exposed to predictable, but not unpredictable, frequency-altered feedback. Participants’ vocal pitch was also measured across each vocalization, to index the extent to which the deviant auditory feedback was used to modify ongoing vocalizations. While both children and adults were found to increase their vocal pitch following predictable and unpredictable changes to their auditory feedback, adults produced larger compensatory responses. The results of the current study demonstrate that both children and adults rapidly integrate information derived from their auditory feedback to modify subsequent speech motor commands. However, these results also demonstrate that children and adults differ in their ability to use auditory feedback to generate compensatory vocal responses during ongoing vocalization. Since vocal variability also differed across the children and adult groups, these results also suggest that compensatory vocal responses to frequency-altered feedback manipulations initiated at vocalization onset may be modulated by vocal variability.

Алгоритм автоматического распознавания речевых команд, инвариантный к изменению языка

Алгоритм автоматического распознавания речевых команд, инвариантный к изменению языка

Construction of Anytime Algorithms for Robust Speech Recognition

Reliability of voice-controlled Ambient Assisted Living (AAL) systems depends mostly on real-time recognition of speech commands coming from a noisy environment. This paper discusses the aspects of constructing anytime algorithms for robust speech recognition in AAL systems. In this case, command recognition is a speaker-dependent classification on a limited set of isolated phrases. An interruptible search can rule out every command that is not a possible solution according to various parameters of speech, and can identify a spoken command with a sufficiently high probability, while keeping time constraints in mind.

Exploration of Alternative Interaction Techniques for Robotic Systems

A robotic system can be considered a computer with the capacity to interact with the real world using sensors and actuators. This article reports on the authors' experiences in developing novel user interfaces for robotic systems, especially in the home environment. High-level control methods that use gestural or speech commands are overly ambiguous or excessively detailed for daily use. The proposed approach is to apply techniques developed in the human-computer interaction field, such as augmented reality and tangible user interfaces, to human-robot interaction. The authors introduce prototype systems such as home appliance control methods using augmented reality, a paper-based method for instructing mobile robots, and various I/O devices to enhance physical interaction with robotic systems. They then discuss lessons learned and directions for future research based on their experiences building these prototype robotic systems and interfaces.