User Commands Research Articles

Few-shot intent detection with mutual information and contrastive learning

Few-shot intent detection is a challenging task. Most existing methods only focus on acquisition of generalization knowledge in known classes, or on the adaptation situation of meta-learning tasks. However, these methods lack both the learning of beneficial information for intent detection and the learning of discriminative representation of intent keywords. Solving these problems can effectively improve the model’s ability to recognize text intent and thus effectively execute user commands. Therefore, in this paper, we propose a few-shot intent detection framework based on Mutual Information and Contrastive Learning (MICL). It improves the robustness of the model learning of intent keywords through mutual information maximization, thus improving the accuracy of intent detection. Additionally, maximization of mutual information enhances the intent representation consistency on two text views,11In this paper, the “two text views” of the text represent the original text and its corresponding data augmentation text.which helps to promote the efficiency of sample contrastive learning. Specifically, we use a neural network to approximately estimate the joint probability distribution and marginal probability distribution of two text views and maximize the mutual information through back-propagation. In order to enable the model to focus on intent keywords in text, we introduce a sample-level contrastive learning. It helps the model to learn discriminative representations of intent keywords by comparing the similarity of text from two text views. In addition, to prevent overfitting of the model on known classes, we introduce a class-level contrastive learning. We regard it as regularization of known classes, aiming to improve the model generalization performance on unknown classes. Our method achieves state-of-the-art performance on four public datasets.22Our method code is available: https://github.com/YS19999/MICL.

Motion planning in underactuated systems with impulsive phenomenon via dynamic shaping of virtual holonomic constraints

Rhythmic motions are traditionally achieved by developing predetermined paths for the states of the space system to follow. Since these paths are obtained offline, the dynamic behavior fails to adapt to changes in environmental conditions or user command desires. The solution we propose is a new strategy called dynamic shaping, in which the paths are formed online to allow the system to create an orbit with the characteristics we need. Hereupon, this paper focuses on applying this strategy to Dynamics with One Degree of Under-actuation and Impulsive Phenomenon (DSODUIP) to adapt the characteristics of outcomes to be in line with the demands.This research was conducted by leveraging the advantages of virtual holonomic constraints (VHCs) to establish these paths. Therefore, a novel two-level hierarchical control method is designed considering a stability criterion to avoid divergence. At the Low-Level, the controllers stabilize the output of system to follow the VHCs on the system. At the High-Level, the VHCs are modified to shape an orbit with our desired characteristic in the motion. As an illustrative example, the algorithm is implemented to adjust the average angular velocity of a devil stick and the hip velocity of a Three-Link biped robot. Their results vividly demonstrate smooth adjustments and efficient performance in achieving our desired outcomes.

Optimization of indoor thermal environment based on sensor networks and multimedia assisted physics teaching

Multimedia technology combines various media elements such as text, images, sound, and video, making classroom teaching more vivid, intuitive, and interesting. This article introduces a multimedia courseware assisted physics teaching application based on sensor networks and deep learning technology. This application has two functional modules: user user end and cluster control end. On the user end, obtain user commands through the virtual machine end and share the original multimedia files using the CIFS protocol. The user command is redirected to the user end, and then the corresponding command is executed on the user end to directly transmit and play the original multimedia file. At the same time, the user end will also provide data feedback and recording for subsequent data analysis and evaluation. At the cluster control end, based on the information collected by the sensor network, an adaptive linear regression algorithm is used to predict the reference value. By analyzing and processing the collected information, the cluster control end can reasonably arrange the playback content of multimedia courseware to meet the learning needs of students. This multimedia courseware assisted physics teaching application based on sensor networks and deep learning technology provides new help and support for physics teaching.

A Wirelessly Powered Scattered Neural Recording Wearable System.

This paper introduces a wirelessly powered scattered neural recording wearable system that can facilitate continuous, untethered, and long-term electroencephalogram (EEG) recording. The proposed system, including 32 standalone EEG recording devices and a central controller, is incorporated in a wearable form factor. The standalone devices are sparsely distributed on the scalp, allowing for flexible placement and varying quantities to provide extensive spatial coverage and scalability. Each standalone device featuring a low-power EEG recording application-specific integrated circuit (ASIC) wirelessly receives power through a 60 MHz inductive link. The low-power ASIC design (84.6 μW) ensures sufficient wireless power reception through a small receiver (Rx) coil. The 60 MHz inductive link also serves as the data carrier for wireless communication between standalone devices and the central controller, eliminating the need for additional data antennas. All these efforts contribute to the miniaturization of standalone devices with dimensions of 12×12×5 mm3, enhancing device wearability. The central controller applies the pulse width modulation (PWM) scheme on the 60 MHz carrier, transmitting user commands at 4 Mbps to EEG recording ASICs. The ASIC employs a novel synchronized PWM demodulator to extract user commands, operating signal digitization and data transmission. The analog frontend (AFE) amplifies the EEG signal with a gain of 45 dB and applies band-pass filtering from 0.03 Hz to 400 Hz, with an input-referred noise (IRN) of 3.62 μVRMS. The amplified EEG signal is then digitized by a 10-bit successive approximation register (SAR) analog-to-digital converter (ADC) with a peak signal-to-noise and distortion ratio (SNDR) of 55.4 dB. The resulting EEG data is transmitted to an external software-defined radio (SDR) Rx through load-shift-keying (LSK) backscatter at 3.75 Mbps. The system's functionality is fully evaluated in human experiments.

Enhancing the Multi-User Experience in Fully Autonomous Vehicles Through Explainable AI Voice Agents

Fully autonomous vehicles (FAVs) operated by AI voice agents create a unique multi-user environment without a traditional driver, allowing users to engage in diverse activities such as entertainment and rest. In addition to the convenience of vehicle control, these agents face the challenge of resolving conflicts arising from multiple simultaneous user commands. This study investigates whether AI voice agents can effectively manage these command conflicts and enhance the multi-user experience (MUX). The research was conducted in two phases: In the preliminary study, online focus group interviews (FGIs) were conducted with 10 participants to explore their perceptions of using AI voice agents in FAVs. Participants shared their experiences and watched a concept video to discuss their expectations of FAV agents. Based on the FGI results, explainable AI voice agents were designed to prevent user conflicts between user commands, focusing on user tendencies and conflict contexts. In the main study, multi-user interactions with these agents were evaluated through online experiments with 89 participants. Two specific experiments were conducted based on the sources of conflict identified in the preliminary study: the first focused on control authority and the second on speech overlap. Participants watched scenario videos and assessed five variables, including sense of agency, trust, problem-solving ability, disappointment, and safety. The findings suggest that explanations provided by AI voice agents can effectively mitigate conflicts and improve MUX in FAVs. However, the effectiveness of these explanations varied across different contexts, indicating the need for alternative approaches. This research provides valuable insights for designing MUXs that prevent multi-user conflicts and meet user expectations in FAV.

Wheelchair Navigation Enhanced by Machine Learning Eye-Gaze Control

Abstract: This study focuses on the development of an eye-controlled wheelchair system tailored for individuals with motor disabilities, with the overarching goal of augmenting their mobility and autonomy. A multifaceted approach was adopted, incorporating meticulous data collection and preprocessing techniques, notably encompassing real-time face detection and feature extraction. Various machine learning algorithms, including KNN, Decision Trees, Random Forests, and SVM, were meticulously trained to anticipate user commands. Through rigorous performance evaluation, the system demonstrated remarkable accuracy and precision, with KNN and SVM emerging as the top-performing models. Notably, this innovative system exhibits significant potential in enhancing accessibility and fostering independence, thereby exemplifying notable strides in the realm of assistive technology. The integration of advanced data processing methodologies, machine learning techniques, and real-time prediction mechanisms facilitates a seamless and intuitive navigation experience, thus holding promise for substantially enriching the quality of life for individuals grappling with mobility disabilities. This research underscores the pivotal role of interdisciplinary collaboration and technological innovation in devising solutions that cater to the diverse needs of individuals with disabilities, thereby advocating for inclusivity and empowerment in society.

A Type System for Flexible User Interactions Handling

Engineering user interfaces involves the use of multiple user interactions. Developers may struggle with programming and using those user interactions because of a lack of flexibility that affects the current user interface programming approaches. First, developers may want to switch from one user interaction to another close one or combine multiple user interactions without changing much code. Second, developers may also want to use several user interactions to concisely produce the same user command. Third, developers may want to be warned about conflicts between involved user interactions. Currently, developers can hardly perform these first two cases without applying numerous code changes or producing boilerplate code. Regarding the third case, developers can only observe such issues during the execution of the interactive systems, which prolongs the detection time. To overcome these three issues this paper proposes a user interaction type system. This user interaction type system reifies user interactions as first-class concerns with typing facilities for enabling user interactions substitutability and union. It also allows the writing of type checking rules to check for possible issues related to user interactions at compile time. We implemented the type system within the TypeScript version of Interacto, a framework for processing user interactions. We evaluated the soundness and the expressiveness of our approach through several implemented use cases. This demonstrates the feasibility of the proposed approach and its ability to overcome the three mentioned issues.

Sensor Based Shopping using Eye-Blink and IoT Technology

The proposed research is an innovative system designed to enhance the shopping experience for physically challenged individuals. The system makes use of relay motors, LCD displays, and Arduino-based eye-blink sensors to provide a user-friendly interface that enables people with physical limitations to explore and engage with shopping environments. Additionally, IoT is used for user interaction. The main functionality of the system is based on eye-blink technology. By detecting eye blinks, the sensors interpret user commands, enabling individuals to control shopping cart movement, and select items from shelves.

Vehicle Control by using Human Voice for Physically Disabled and Aged People

The most crucial form of human communication is speech signals. Almost all encounters use speech signals in conversations. With a microphone, sounds and different speech signals can be transformed into electrical form. Physical disability can result from a variety of causes, such as age-related issues, health issues, and accident-related injuries. For these challenged individuals who suffer from impairments in their hands and legs, vehicles are used as a means of transportation. It might be challenging for anyone with certain conditions, such as paralysis, to manually push a wheelchair or use a remote assembly. For these individuals, the project is made to work with voice-based commands, enabling the paralyzed or impaired person to offer directions simply by speaking into the provided microphone. Additionally, the device has directional buttons for remote vehicle control. An Atmega328-based circuit interfaced with a voice recognition module builds up the system. The voice recognition module receives user commands and translates them into digital data, which the micro-controller analyzes to obtain directional orders. The transmitter circuit and the receiver circuit make up the whole system. The voice recognition module is part of the transmitter circuit, and the motor and driver assembly are part of the one that receives circuit. For communication, an NRF trans-receiver module is used. The command issued to the vehicle is shown on a 16*2 LCD. Keywords: Bluetooth module, Arduino, wireless night vision camera, robotic arm.

An Approach Towards to Real Time AI Desktop Voice Assistant

The advent of artificial intelligence (AI) has revolutionized human-computer interaction, making it more natural and intuitive. This research paper presents the development and implementation of an AI Desktop Voice Assistant designed to enhance productivity and accessibility for users. The voice assistant leverages advanced speech recognition, natural language processing (NLP), and machine learning techniques to understand and execute user commands. Key functionalities include voice-activated application control, web searches, personalized reminders, and real-time information retrieval. Our system integrates with widely-used APIs and services, providing a seamless user experience across various tasks. The paper discusses the architecture of the voice assistant, detailing the integration of components such as the speech recognition engine, NLP models, and the dialogue management system. We explore the challenges encountered during development, including accurate speech recognition in noisy environments and handling ambiguous user commands. Solutions implemented to address these challenges are also presented. Furthermore, we conduct a usability study to evaluate the effectiveness and user satisfaction of the voice assistant. The results indicate a high level of user engagement and satisfaction, demonstrating the practical benefits of incorporating AI-driven voice interfaces into desktop environments. Our findings contribute to the ongoing research in human-computer interaction, suggesting pathways for future enhancements in AI voice assistants..

Smart Home Automation System

This project introduces the overall framework of an economical wireless home automation system (HAS). Its primary emphasis lies in establishing an Internet of Things (IoT)-centered home automation setup, capable of automatic configuration based on environmental conditions and capable of overseeing multiple devices remotely via the internet. The objective is to design firmware for intelligent control, ensuring automated functionality with minimal human intervention to uphold the well-being of all electrical appliances in the household. Node MCU, a widely recognized open-source IoT platform, has been employed for the automation process. Various transmission modes are implemented across distinct system components to relay user commands from Node MCU to the actual device. The central control system utilizes wireless technology, enabling remote accessibility via a smartphone. The incorporation of a cloud server-based communication system enhances project practicality, providing users unrestricted access to appliances, regardless of their spatial proximity. To fortify automation, a data transmission network has been established. The user-friendly interface, cost-effective construction, and straightforward installation make it an ideal solution for managing household electrical appliances and devices. Both control and appliance status can be monitored through an Android platform. The system's objective is to enhance the living conditions of elderly and disabled individuals by offering necessary support and assistance, ultimately elevating the overall home living experience with the concept of a smart home.

Lipwatch: Enabling Silent Speech Recognition on Smartwatches using Acoustic Sensing

Silent Speech Interfaces (SSI) on mobile devices offer a privacy-friendly alternative to conventional voice input methods. Previous research has primarily focused on smartphones. In this paper, we introduce Lipwatch, a novel system that utilizes acoustic sensing techniques to enable SSI on smartwatches. Lipwatch leverages the inaudible waves emitted by the watch's speaker to capture lip movements and then analyzes the echo to enable SSI. In contrast to acoustic sensing-based SSI on smartphones, our development of Lipwatch takes into full consideration the specific scenarios and requirements associated with smartwatches. Firstly, we elaborate a wake-up-free mechanism, allowing users to interact without the need for a wake-up phrase or button presses. The mechanism utilizes the inertial sensors on the smartwatch to detect gestures, in combination with acoustic signals that detecting lip movements to determine whether SSI should be activated. Secondly, we design a flexible silent speech recognition mechanism that explores limited vocabulary recognition to comprehend a broader range of user commands, even those not present in the training dataset, relieving users from strict adherence to predefined commands. We evaluate Lipwatch on 15 individuals using a set of the 80 most common interaction commands on smartwatches. The system achieves a Word Error Rate (WER) of 13.7% in user-independent test. Even when users utter commands containing words absent in the training set, Lipwatch still demonstrates a remarkable 88.7% top-3 accuracy. We implement a real-time version of Lipwatch on a commercial smartwatch. The user study shows that Lipwatch can be a practical and promising option to enable SSI on smartwatches.

Islamic QA with Chatbot System Using Convolutional Neural Network

Many questions and answers about Islamic law are scattered on the internet and have been explained repeatedly by various sites. One solution is presented by the website www.piss-ktb.com, which creates a web-based source of information in the form of Frequently Asked Questions (FAQ). However, web-based FAQs have a weakness: users still have to browse through the available questions one by one according to the questions they want to know the answers to. Browsing through thousands of FAQs is inefficient and exhausting. Thus, a chatbot system can become a better alternative to the FAQ website. Still, chatbots are difficult to use because most of their conversations are hard to understand. A single character error will cause the system to misunderstand its meaning. In reality, users expect a chatbot that can understand everyday language. Thus, it is necessary to develop a chatbot system that can understand various common sentence combinations in everyday language and understand the meaning of words. In addition, it should be able to predict answers automatically to various kinds of questions and requests, even though the initial training data is relatively low. Therefore, this study aims to develop a system that can provide answers automatically based on user commands in natural language using Global Vectors for Word Representations (GloVe), Convolutional Neural Networks (CNN), and Transfer Learning techniques. The result shows that the use of transfer learning and the Nadam optimizer can improve the system’s performance.

Efficient Feature Learning Model of Motor Imagery EEG Signals with L1-Norm and Weighted Fusion.

Brain-computer interface (BCI) for motor imagery is an advanced technology used in the field of medical rehabilitation. However, due to the poor accuracy of electroencephalogram feature classification, BCI systems often misrecognize user commands. Although many state-of-the-art feature selection methods aim to enhance classification accuracy, they usually overlook the interrelationships between individual features, indirectly impacting the accuracy of feature classification. To overcome this issue, we propose an adaptive feature learning model that employs a Riemannian geometric approach to generate a feature matrix from electroencephalogram signals, serving as the model's input. By integrating the enhanced adaptive L1 penalty and weighted fusion penalty into the sparse learning model, we select the most informative features from the matrix. Specifically, we measure the importance of features using mutual information and introduce an adaptive weight construction strategy to penalize regression coefficients corresponding to each variable adaptively. Moreover, the weighted fusion penalty balances weight differences among correlated variables, reducing the model's overreliance on specific variables and enhancing accuracy. The performance of the proposed method was validated on BCI Competition IV datasets IIa and IIb using the support vector machine. Experimental results demonstrate the effectiveness and superiority of the proposed model compared to the existing models.

IOT Based DC Motor Speed and Direction Control

This paper introduces an innovative Internet of Things (IoT) based system designed to control the speed and direction of a DC motor using the Blynk mobile application. The system integrates an OLED display (SSD1306), an ESP8266 microcontroller, a 300 RPM DC motor, and an L293D motor driver. With this system, users can remotely manage the speed and direction of the DC motor through the Blynk app, accessible on smartphones or tablets. Real-time feedback on the motor's speed and direction is provided via the OLED display, enhancing user interaction and experience. The ESP8266 microcontroller acts as the core control unit, interfacing with the Blynk app via Wi-Fi to receive user commands and adjust the motor's operation accordingly. The bidirectional control feature of the L293D motor driver enables precise manipulation of the motor's speed and direction. By leveraging IoT technology and the Blynk app, users gain the flexibility to control the motor from any location with internet connectivity, offering convenience and accessibility. Additionally, the OLED display enhances the user interface, enabling intuitive monitoring and adjustment of motor parameters. In conclusion, this system offers a versatile and user-friendly solution for remotely managing DC motor speed and direction, with potential applications spanning across home automation, robotics, and industrial control systems.

Design of an Assisted Driving System for Obstacle Avoidance Based on Reinforcement Learning Applied to Electrified Wheelchairs

Driving a motorized wheelchair is not without risk and requires high cognitive effort to obtain good environmental perception. Therefore, people with severe disabilities are at risk, potentially lowering their social engagement, and thus, affecting their overall well-being. Therefore, we designed a cooperative driving system for obstacle avoidance based on a trained reinforcement learning (RL) algorithm. The system takes the desired direction and speed from the user via a joystick and the obstacle distribution from a LiDAR placed in front of the wheelchair. Considering both inputs, the system outputs a pair of forward and rotational speeds that ensure obstacle avoidance while being as close as possible to the user commands. We validated it through simulations and compared it with a vector field histogram (VFH). The preliminary results show that the RL algorithm does not disruptively alter the user intention, reduces the number of collisions, and provides better door passages than a VFH; furthermore, it can be integrated on an embedded device. However, it still suffers from higher jerkiness.

Arduino Based Pickup and Drop Robot

A robotic arm is designed using Arduino to pick and place the objects via user commands. It will pick and place an object from source to destination safely. The soft catching gripper used in the arm will not apply any extra pressure on the objects. The robot is controlled using android based smartphones through Bluetooth. Based on the commands given by the user the robot moves accordingly. At the receiver end there are four motors interfaced with the micro controller. Two for the vehicle movement and the remaining two are for arm and gripper movement. Blue control application is used for the controlling of robot Keywords: Six Axis, Bluetooth Module, Micro-controller, Stepper Motor.

Sasha

Smart home assistants function best when user commands are direct and well-specified---e.g., "turn on the kitchen light"---or when a hard-coded routine specifies the response. In more natural communication, however, human speech is unconstrained, often describing goals (e.g., "make it cozy in here" or "help me save energy") rather than indicating specific target devices and actions to take on those devices. Current systems fail to understand these under-specified commands since they cannot reason about devices and settings as they relate to human situations. We introduce large language models (LLMs) to this problem space, exploring their use for controlling devices and creating automation routines in response to under-specified user commands in smart homes. We empirically study the baseline quality and failure modes of LLM-created action plans with a survey of age-diverse users. We find that LLMs can reason creatively to achieve challenging goals, but they experience patterns of failure that diminish their usefulness. We address these gaps with Sasha, a smarter smart home assistant. Sasha responds to loosely-constrained commands like "make it cozy" or "help me sleep better" by executing plans to achieve user goals---e.g., setting a mood with available devices, or devising automation routines. We implement and evaluate Sasha in a hands-on user study, showing the capabilities and limitations of LLM-driven smart homes when faced with unconstrained user-generated scenarios.

Design and Development of Automated Control of Restaurant Robot

This paper presents a novel voice-controlled restaurant robot system designed to revolutionize the dining experience by seamlessly integrating advanced robotics and natural language processing technologies. The proposed system aims to enhance customer service, operational efficiency, and overall dining satisfaction by providing a user-friendly and interactive platform. .The restaurant robot is equipped with a comprehensive set of capabilities, including, voice recognition and manipulation. By leveraging voice control, users can interact with the robot using natural language commands, enabling seamless communication and intuitive control. This allows customers to effortlessly place orders The system's is comprises a mobile robotic platform, bluetooth modules, a cloud-based natural language processing, The voice control functionality is implemented through a sophisticated speech recognition system, enabling the robot to accurately understand and interpret user commands. .In addition to customer-facing interactions, the restaurant robot also assists restaurant staff in various tasks. Deliver food and beverages to tables, handle payment transactions, and perform routine cleaning tasks. By automating these processes, the system improves operational efficiency, reduces labor costs, and allows human staff to focus on highervalue activities such as customer engagement and culinary expertise. .To evaluate the performance and usability of the voice-controlled restaurant robot, a series of experiments were conducted in a real-world restaurant environment. The results demonstrated the system's high recognition accuracy, robustness in noisy environments, and its ability to effectively handle a wide range of user requests. .The proposed voice-controlled restaurant robot presents a promising solution for transforming the dining experience. It combines the convenience of voice control with advanced robotics, enabling seamless and interactive interactions between customers, staff, and the robotic platform. The system's ability to enhance customer service, operational efficiency, and overall dining satisfaction positions it as a significant advancement in the hospitality industry.

CLARA: Classifying and Disambiguating User Commands for Reliable Interactive Robotic Agents

CLARA: Classifying and Disambiguating User Commands for Reliable Interactive Robotic Agents