Command Recognition Research Articles

Reliable, efficient, and scalable photonic inverse design empowered by physics-inspired deep learning

Abstract On-chip computing metasystems composed of multilayer metamaterials have the potential to become the next-generation computing hardware endowed with light-speed processing ability and low power consumption but are hindered by current design paradigms. To date, neither numerical nor analytical methods can balance efficiency and accuracy of the design process. To address the issue, a physics-inspired deep learning architecture termed electromagnetic neural network (EMNN) is proposed to enable an efficient, reliable, and flexible paradigm of inverse design. EMNN consists of two parts: EMNN Netlet serves as a local electromagnetic field solver; Huygens–Fresnel Stitch is used for concatenating local predictions. It can make direct, rapid, and accurate predictions of full-wave field based on input fields of arbitrary variations and structures of nonfixed size. With the aid of EMNN, we design computing metasystems that can perform handwritten digit recognition and speech command recognition. EMNN increases the design speed by 17,000 times than that of the analytical model and reduces the modeling error by two orders of magnitude compared to the numerical model. By integrating deep learning techniques with fundamental physical principle, EMNN manifests great interpretability and generalization ability beyond conventional networks. Additionally, it innovates a design paradigm that guarantees both high efficiency and high fidelity. Furthermore, the flexible paradigm can be applicable to the unprecedentedly challenging design of large-scale, high-degree-of-freedom, and functionally complex devices embodied by on-chip optical diffractive networks, so as to further promote the development of computing metasystems.

Speech-Controlled Robot Enabling Cognitive Training and Stimulation in Dementia Prevention for Severely Disabled People

Abstract The current German medical S3-guideline on dementia recommends the use of cognitive training and stimulation, e.g. through shared games, for mild cognitive impairment and mild dementia. The use of cobots, which allow direct human-robot collaboration, enables people with paralyzed upper limbs to actively participate in social activities. This research presents a novel approach through the development of a voice-controlled board game specifically tailored to the inclusion needs of people with severe disabilities. The human voice commands recorded via USB microphones are digitally filtered. Speech recognition of the control commands is performed using a Convolutional Neural Network (CNN) based on the VGG- 16 architecture. The robot´s activity is controlled utilizing the ROS 2 robot operating system. A portable table-based complete system with a 3D-printed Tic-Tac-Toe playing field and robot assistant for severely disabled paralyzed people has been developed. The robot control system employs a pick-and-place mechanism, seamlessly integrating with speech recognition to enhance gameplay interaction. The CNN model achieves an impressive accuracy rate of 97%, ensuring reliable speech recognition performance throughout gameplay. The targeted integration of robotic technologies and artificial intelligence opens new avenues in the prevention of mental illness, care support and inclusion of older and disabled people.

Phoneme-by-Phoneme Speech Recognition as a Classification of Series on a Set of Sequences of Elements of Complex Objects Using an Improved Trie-Tree

Sequences, including vector sequences, are applicable in any subject domains. Sequences of scalar values or vectors (series) can be produced by higher-order sequences, for example: a series of states, or elements of complex objects. This academic paper is devoted to the application of an improved trie-tree in the classification of series on a set of sequences of elements of complex objects using the dynamic programming method. The implementation areas of dynamic programming have been considered. It has been shown that dynamic programming is adapted to multi-step operations of calculating additive (multiplicative) similarity/difference measures. It is argued that the improved trie-tree is applicable in the problem of classifying a series on a set of sequences of elements of complex objects using such similarity/difference measures. An analysis of hierarchical representations of sets of sequences has been performed. The advantages of the improved trie-tree over traditional representations of other highly branching trees have been described. A formal description of the improved trie-tree has been developed. An explanation has been given to the previously obtained data on a significant speed gain for operations of adding and deleting sequences in the improved trie-tree relative to the use of an array with an index table (24 and 380 times, respectively). The problem of phoneme-by-phoneme recognition of speech commands has been formulated as a problem of classifying series on a set of sequences of elements of complex objects and a method for its solving has been presented. A method for classifying a series on a set of sequences of elements of complex objects using the improved trie-tree is developed. The method has been studied using the example of phoneme-by-phoneme recognition with a hierarchical representation of the dictionary of speech command classes. In this method, recognition of speech commands is executed traversing the improved trie-tree that stores a set of transcriptions of speech commands – sequences of transcription symbols that denote classes of sounds. Numerical studies have shown that classifying a series as sequences of elements of complex objects increases the frequency of correct classification compared to classifying a series on a set of series, and using the improved trie-tree reduces the time spent on classification.

Design of voice command recognition chip based on heterogeneous acceleration

Abstract Speech, as one of the earliest forms of communication used by humans, can effectively convey information. However, the current deep neural network models for speech recognition are generally large in scale and can only be deployed in the cloud, which imposes high deployment environment requirements and power consumption, thereby limiting their implementation on embedded devices. In the context of end-to-end speech recognition, a series of challenges are encountered, including power consumption constraints, computing power limitations, network dependencies, privacy protection, bandwidth restrictions, and communication delays. To address these issues, this paper proposes the design of an end-to-end voice command recognition chip based on deep neural networks specifically for recognizing voice commands in specific scenarios. This chip achieves low power consumption and minimal delay in recognition. Additionally, we introduce a weighted, overloadable chip architecture to enable seamless scene migration, ultimately aiming to resolve the aforementioned challenges.

Application of Deep Learning for Voice Command Classification in Turkish Language

In this study, a deep learning model was developed for the recognition and classification of voice commands using the Turkish Speech Command Dataset. The division of training, validation, and test sets was carried out on an individual basis. This approach aims to prevent the model from memorizing and to enhance its generalization capability. The model was trained using Mel-Frequency Cepstral Coefficients (MFCC) features extracted from voice files, and its classification performance was evaluated in detail. The findings indicate that the model successfully classifies voice commands with a high accuracy rate, achieving an overall accuracy of 92.3% on the test set, highlighting the potential of deep learning approaches in voice recognition technologies

Robust speech command recognition in challenging industrial environments

Speech is among the main forms of communication between humans and robots in industrial settings, being the most natural way for a human worker to issue commands. However, the presence of pervasive and loud environmental noise poses significant challenges to the adoption of Speech-Command Recognition systems onboard manufacturing robots; indeed, they are expected to perform in real time on hardware with limited computational capabilities and also to be robust and accurate in such complex environments. In this paper, we propose an innovative system based on an End-to-End architecture with a Conformer backbone. Our system is specifically designed to achieve high accuracy in noisy industrial environments and to guarantee a minimal computational burden to meet stringent real-time requirements while running on computing devices that are embedded in robots. In order to increase the generalization capability of the system, the training procedure is driven by a Curriculum Learning strategy combined with dynamic data augmentation techniques, that progressively increase the complexity of input samples by increasing the noise during the training phase. We have conducted extensive experimentation to assess the effectiveness of our system, using a dataset composed of more than 50,000 samples, of which about 2,000 have been acquired during the daily operations of a Stellantis Italian factory. The results confirm the suitability of the proposed approach to be adopted in a real industrial environment; indeed, it is able to achieve, on both English and Italian commands, an accuracy higher than 90%, maintaining a compact model size (the network is 1.81 MB) and running in real-time on an industrial embedded device (namely 41ms over an NVIDIA Xavier NX).

Investigation of attention mechanism for speech command recognition

Investigation of attention mechanism for speech command recognition

Bio-inspired EEG signal computing using machine learning and fuzzy theory for decision making in future-oriented brain-controlled vehicles

One kind of autonomous vehicle that can take instructions from the driver by reading their electroencephalogram (EEG) signals using a Brain-Computer Interface (BCI) is called a Brain-Controlled Vehicle (BCV). The operation of such a vehicle is greatly affected by how well the BCI works. At present, there are limitations on the accuracy of BCI recognition, the number of distinguishable command categories, and the execution duration of command recognition. Consequently, vehicles that are exclusively controlled by EEG signals demonstrate suboptimal control performance. To address the difficulty of improving the control capabilities of brain-controlled cars while maintaining BCI performance, a fuzzy logic-based technique called as Fuzzy Brain-Control Fusion Control is introduced. This approach uses Fuzzy Discrete Event System (FDES) supervisory theory to verify the accuracy of the driver's brain-controlled directives. Concurrently, a fuzzy logic-based automatic controller is developed to generate decisions automatically in accordance with the present state of the vehicle via fuzzy reasoning. The final decision is then reached through the application of secondary fuzzy reasoning to the accuracy of the driver's instructions and the automated decisions to make adjustments that are more consistent with human intent. A clever BCI gadget known as the Consistent State Visual Evoked Potential (SSVEP) is utilized to show the viability of the proposed technique. We recommend that additional research should be conducted at this time to confirm that our recommended system may further improve the control execution of BCI-fueled cars, regardless of whether BCIs have special limitations.

CNN Based Automatic Speech Recognition: A Comparative Study

Recently, one of the most common approaches used in speech recognition is deep learning. The most advanced results have been obtained with speech recognition systems created using convolutional neural network (CNN) and recurrent neural networks (RNN). Since CNNs can capture local features effectively, they are applied to tasks with relatively short-term dependencies, such as keyword detection or phoneme- level sequence recognition. This paper presents the development of a deep learning and speech command recognition system. The Google Speech Commands Dataset has been used for training. The dataset contained 65.000 one-second-long words of 30 short English words. That is, %80 of the dataset has been used in the training and %20 of the dataset has been used in the testing. The data set consists of one-second voice commands that have been converted into a spectrogram and used to train different artificial neural network (ANN) models. Various variants of CNN are used in deep learning applications. The performance of the proposed model has reached %94.60.

Significance of chirp MFCC as a feature in speech and audio applications

A novel feature, based on the chirp z-transform, that offers an improved representation of the underlying true spectrum is proposed. This feature, the chirp MFCC, is derived by computing the Mel frequency cepstral coefficients from the chirp magnitude spectrum, instead of the Fourier transform magnitude spectrum. The theoretical foundations for the proposal, and the experimental validation using product of likelihood Gaussians, to show the improved class separation offered by the proposed chirp MFCC, when compared with basic MFCC are discussed. Further, real world evaluation of the feature is performed using three diverse tasks, namely, speech–music classification, speaker identification, and speech commands recognition. It is shown in all three tasks that the proposed chirp MFCC offers considerable improvements.

End-to-end elevator voice command recognition system based on improved convolutional neural network

Abstract As a common life scenario, the use of voice control in elevator operation can effectively reduce public health risks and improve user experience in the post-epidemic era. However, due to the characteristics of its closure and public space, it is difficult to deploy the elevator. To solve this problem, an end-to-end speech command recognition algorithm based on a new deep neural network is proposed. The algorithm uses the self-built corpus for training, and uses the speech segment interception algorithm to obtain audio segments under the premise of speech stream as input, puts them into the network model in real time for reasoning, and drives the elevator to run according to the output results. The results show that compared with the recognition model using MFCC-CNN, the network has achieved about 10% improvement in accuracy under the premise of smaller computation, and compared with the DTW algorithm, the network has improved the accuracy by about 25%. Finally, a typical deployment environment is constructed to prove the correctness and effectiveness of the method in practical application.

A Case Study on (EAVIBA) Enhanced Accessibility for Visually Impaired & Blind Artists using Machine Learning

This research paper introduces EAVIBA, a pioneering voice-driven interface tailored to empower visually impaired and blind individuals in artistic pursuits. Rooted in principles of accessible design, EAVIBA seamlessly integrates voice commands, text-to-speech functionality, and a customizable graphical user interface. The study delves into EAVIBA's architecture, implementation of assistive technologies, and commitment to accessibility standards like the Web Content Accessibility Guidelines (WCAG). Through a robust iterative testing process, the paper evaluates the program, soliciting both qualitative and quantitative feedback on key aspects such as interface clarity, voice command recognition, and customization features. Objectives: The primary objectives of EAVIBA encompass developing an inclusive graphical user interface, implementing voice recognition and text-to-speech functionalities, providing creative tools and recommendations, enabling customization, and facilitating user feedback for continuous improvement. Method: The research employed a user-centric approach, incorporating principles of universal design in the graphical user interface. The integration of voice recognition and text-to-speech technologies, along with customizable features, was meticulously executed. Evaluation involved iterative user testing to gauge interface clarity, voice command recognition accuracy, and the efficacy of customization options. Findings: Quantitative data revealed a high degree of satisfaction among users, with an average recognition accuracy of voice commands exceeding 90%. Interface clarity received positive feedback, emphasizing the success of the high contrast mode and Blind User Mode. Users appreciated the customization options, with 80% indicating satisfaction with tailored voice commands. Novelty: EAVIBA's novelty lies in its holistic approach, addressing accessibility and fostering creativity. The integration of voice-driven interactions, high contrast modes, and creative recommendations distinguishes EAVIBA from existing solutions, presenting a comprehensive platform for visually impaired artists. Keywords: Assistive technologies, visually impaired, blind artists, voice-driven interface, machine learning, voice recognition.

Enhancing Command Recognition in Air Traffic Control Through Advanced Classification Techniques

ABSTRACT This paper addresses the persistent challenges in speech processing within the Air Traffic Control (ATC) domain, a field where despite extensive research, issues such as handling noisy environments, accented speech, and the need for strict adherence to standard phraseology continue to undermine conventional language models. Our study employs a hybrid approach that integrates syntactic analysis with advanced machine learning classification algorithms – Logistic Regression, Lagrangian Support Vector Machine, and Naïve Bayes. By mixing and matching algorithms tailored for specific aspects of speech processing, our approach moves away from traditional reliance on a singular integrated system, illustrating through rigorous testing with the ATCOSIM dataset that such a multifaceted strategy markedly improves command recognition accuracy and adapts more effectively to the unique linguistic features of ATC speech. Results highlight the superior performance of Logistic Regression across various command recognition categories, pointing towards a promising direction for future advancements in ATC speech recognition technologies aimed at reducing human workload and increasing automation precision. This paper explores the complexities of the required analysis techniques and underscores the necessity of employing diverse algorithms in the processing pipeline to enhance overall system accuracy.

Audio–visual speech recognition based on regulated transformer and spatio–temporal fusion strategy for driver assistive systems

This article presents a research methodology for audio–visual speech recognition (AVSR) in driver assistive systems. These systems necessitate ongoing interaction with drivers while driving through voice control for safety reasons. The article introduces a novel audio–visual speech command recognition transformer (AVCRFormer) specifically designed for robust AVSR. We propose (i) a multimodal fusion strategy based on spatio–temporal fusion of audio and video feature matrices, (ii) a regulated transformer based on iterative model refinement module with multiple encoders, (iii) a classifier ensemble strategy based on multiple decoders. The spatio–temporal fusion strategy preserves contextual information of both modalities and achieves their synchronization. An iterative model refinement module can bridge the gap between acoustic and visual data by leveraging their impact on speech recognition accuracy. The proposed multi-prediction strategy demonstrates superior performance compared to traditional single-prediction strategy, showcasing the model’s adaptability across diverse audio–visual contexts. The transformer proposed has achieved the highest values of speech command recognition accuracy, reaching 98.87% and 98.81% on the RUSAVIC and LRW corpora, respectively. This research has significant implications for advancing human–computer interaction. The capabilities of AVCRFormer extend beyond AVSR, making it a valuable contribution to the intersection of audio–visual processing and artificial intelligence.

Enhanced Accessibility for Visually Impaired & Blind Artists (EAVIBA) using Machine Learning

This research paper introduces EAVIBA, an innovative voice-driven interface aimed at empowering individuals with visual impairments and blindness in the domain of art and creativity. Rooted in principles of accessible design, EAVIBA integrates advanced features including voice commands, text-to-speech functionality, and a customizable graphical user interface, exemplifying a commitment to inclusive technology. The paper delves into the intricacies of EAVIBA's development, elucidating its architectural framework, seamless integration of assistive technologies, and adherence to recognized accessibility standards such as the Web Content Accessibility Guidelines (WCAG). Employing a comprehensive approach, the research paper outlines the iterative process of user testing, providing both qualitative and quantitative insights into critical aspects such as interface clarity, voice command recognition accuracy, and the efficacy of customization options. The results of these evaluations affirm EAVIBA's remarkable success in delivering an inclusive, accessible, and empowering environment for individuals with visual impairments to engage in artistic expression, thereby contributing significantly to the broader landscape of assistive technologies and accessible interfaces. Keywords: Visually impaired, blind artists, voice-driven interface, assistive technologies, voice recognition, machine learning.

VIRTUAL MOUSE USING EYE MOVEMENTS FOR HANDICAPPED

A disabled person’s life is always dependent on someone else who needs aid with mobility or any other task. Individuals with disabilities may face challenges when using computers. The most common way of interacting with computers is using a mouse and keyboard. It is difficult for people with physical disabilities to use them. Facial movements and voice commands are one of the best possible actions by physically disabled individuals, by recognizing and responding to these movements and commands , it is possible for them to operate the computer using only their eye movements. Eye movement recognition and voice command is a contemporary approach to interaction between humans and computers. The proposed system can easily control the computers by using eye movement and voice command recognition. It can be a viable replacement for traditional system in the future. This research outlines the techniques utilized in the design, implementation, and evaluation of the experiments conducted and presents the results obtained. keyword - Email, Internet, Voice, Speech recognition, physically challenged, Text to speech, Blind, eye movements .

Optimizing Speech to Text Conversion in Turkish: An Analysis of Machine Learning Approaches

The Conversion of Speech to Text (CoST) is crucial for developing automated systems to understand and process voice commands. Studies have focused on developing this task, especially for Turkish-specific voice commands, a strategic language in the international arena. However, researchers face various challenges, such as Turkish's suffixed structure, phonological features and unique letters, dialect and accent differences, word stress, word-initial vowel effects, background noise, gender-based sound variations, and dialectal differences. To address the challenges above, this study aims to convert speech data consisting of Turkish-specific audio clips, which have been limitedly researched in the literature, into texts with high-performance accuracy using different Machine Learning (ML) models, especially models such as Convolutional Neural Networks (CNNs) and Convolutional Recurrent Neural Networks (CRNNs). For this purpose, experimental studies were conducted on a dataset of 26,485 Turkish audio clips, and performance evaluation was performed with various metrics. In addition, hyperparameters were optimized to improve the model's performance in experimental studies. A performance of over 97% has been achieved according to the F1-score metric. The highest performance results were obtained with the CRNN approach. In conclusion, this study provides valuable insights into the strengths and limitations of various ML models applied to CoST. In addition to potentially contributing to a wide range of applications, such as supporting hard-of-hearing individuals, facilitating notetaking, automatic captioning, and improving voice command recognition systems, this study is one of the first in the literature on CoST in Turkish.

ThumbUp: Secure Smartwatch Controller for Smart Homes Using Simple Hand Gestures

The development of creative applications and intelligent gadgets requires a secure and straightforward interface with human users. We propose, design, and implement ThumbUp, a smartwatch-based two-factor real-time identification and authentication system in which smartwatch users can identify and authenticate themselves using some simple hand and finger movements, such as thumb-up. ThumbUp leverages the signal from the Inertial Measurement Unit (IMU) in in Commercial-Off-The-Shelf (COTS) smart devices to discover the unique pattern generated by each user's simple gestures using a carefully constructed deep learning model. Smart homes provide a comfortable, safe, and efficient living environment, epecially help the sick and aged. We propose strategies for convenient and reliable control in smart homes with gesture command recognition. We build an Auto-Encoder-based filter that reconstructs the raw data to improve the representation of gesture features. Moreover, we adopt the random forest method to analyse the contextual command correlation. And we employ the authentication system based on smartwatch for personalized command feedback and ensure that illegals cannot use the device. We implement our system and undertake rigorous studies to determine its usefulness and efficiency over a three-month period with 65 users. It achieves a 97% accuracy for user classification and an EER of 0.014 for authentication task with a single simple gesture. And our method achieves 91% accuracy for command recognition and 96% command accuracy with contextual informations. Additionally, we conduct a study of user acceptability of our system and explain how gesture proficiency influences authentication accuracy.

Mordo2: A Personalization Framework for Silent Command Recognition.

Wearable human-computer interactions in daily life are increasingly encouraged by the prevalence of intelligent wearables. It poses a demanding requirement of micro-interaction and minimizing social awkwardness. Our previous work demonstrated the feasibility of recognizing silent commands through around-ear biosensors with the limitation of user adaptation. In this work, we ease the limitation by a personalization framework that integrates spectral factorization of signals, temporal confidence rejection and commonly used transfer learning algorithms. Specifically, we first empirically formulate the user adaptation issue by presenting the accuracies of applying transfer learning algorithms to our previous method. Second, we improve the signal-to-noise ratio by proposing the supervised spectral factorization method that learns the amplitude and phase mappings between around-ear signals and the signals of articulated facial muscles. Third, we leverage the time continuity of commands and introduce the time decay into confidence rejection. Finally, extensive experiments have been conducted to evaluate the feasibility and improvements. The results indicate an average accuracy of 92.38% which is significantly larger than solely using transfer learning algorithms. And a comparable accuracy can be achieved with significantly reduced data of new users. The overall performance shows the framework can significantly improve the accuracy of user adaptations. The work would aid a further step toward commercial products for silent command recognition and inspire the solution to the user adaptation challenge of wearable human-computer interactions.

Perancangan Sistem Deteksi Dan Pengenalan Perintah Suara Menggunakan Modul Esp 32 Dengan Metode Convolutional Neural Network (Cnn)

ABSTRACT
 The capabilities of microcontrollers are currently developing rapidly. Thanks to Moore's Law, the number of embedded transistors is growing exponentially. This has led to increasingly sophisticated microcontroller capabilities that are inversely proportional to price, so that we can now embed artificial intelligence on microcontrollers with the help of one of Google's APIs, TensorFlow Lite. In this final project research, a voice command detection and recognition system will be designed using a 32-bit microcontroller in the form of an ESP32 module with WiFi connectivity as the main processor to perform data processing and recognition in the form of voice commands. Voice input is taken using a MEMS (Micro Electro Mechanical System) microphone using the I2C interface. The data sent has a width of 8 bits with a sampling frequency of 44 kHz. The sampling data results will be used in the artificial intelligence training and testing process, together with a voice command library collection with a data size of 4 GB consisting of 20 word files in English, Indonesian, and noise or environmental background samples. Training and testing are carried out by converting the results of the voice signal sampling data into voice spectrum images for input to the CNN (Convolutional Neural Network) algorithm. The results of the experiment show that the voice command recognition process has an accuracy of 90% with a recognition time of less than 1 second.
 Keyword : esp32,Voice, CNN, MEMS, tensorflow Lite