Command Input Research Articles

Multimodal Smart Wheelchair Integrated with Safety Alert System

Elderly and people with disabilities often rely on others for their locomotion. With this emerging world of automation and technologically advanced society we live in a smart wheelchair with appropriate automation can be a life changing innovation for them. One might wonder what is the need for a wheelchair to be smart or for that matter why anything needs to be smart. The answer is simple, to overcome the limitations of the existing technology. We aim on integrating a simple Manually operated wheelchair with features like Obstacle Detection with appropriate technologies such as voice control or gesture control for people who are not able to locomote like a normal person can. With this project, not only do we help a person using a normal wheelchair more easily but also make life easier for those who have other disabilities which are standing as an obstacle making it difficult for them to walk. For instance, a blind person can use a smart wheelchair that allows voicecontrolled movement or even gesture-controlled movement. Another might be of a person who can’t speak, will now be able to control everything with his bare hands. Wheelchair coupled with the appropriate sensors that automatically detects the obstacles/objects in the proximity and takes appropriate action consequently and In addition to that it may also be controlled by another person taking care of the disabled by giving commands such as forward, backward, upright etc. This not only reduces the user's efforts but also helps people to take care of their elderly. Voice/gesture control system makes everything simple. Just visualize the application of it in a hospital where the nurse has to manually handle the patient's wheelchair for even the slightest of movement. This system on the other hand needs only text or voice input command, and based upon the predefined command received from the user, the system will execute the task. This project even enforces a GSM module which uses the sim card, which can help in tracking the wheelchair when required, like in case where a user is in difficulty and needs emergency help, a message asking for help can be sent to the intended person.

A Method for Generating Variable-Scale Maps for Small Displays

With the rapid development of the internet and information technology, visualization techniques for mobile and interactive web maps have developed different requirements. Small screens make it difficult to simultaneously present information details and the surrounding context. Aiming at this problem, this paper proposes a novel variable-scale method that can allow users to properly specify the size, shape, and number of the focus area(s). Our method first establishes a hierarchical data structure for representing geographic data and then the client-side can request and represent the information according to only the operational command input by users. Experimental results show that this method can realize the variable-scale representation of real geographic data on a single screen. It can effectively solve the contradiction between a small-screen display and a large quantity of information.

Smart Home with Biometric System Recognition

In the last decade, smart homes have become applicable and widely developed where all the electronic devices and household appliances are controlled from a centralized control unit. However, the rise of misuses such as the security aspect and interoperability mechanism is proving them to be less reliable. Moreover, the traditional home automation system is complex, expensive and they are inconvenient for the aging population or disable persons. Besides, most smart home technologies forcing users to use their hands in order to control electronic devices. Therefore, the smart home with biometric system recognition is proposed to increase security, efficiency and decrease manual labor effort. This project used biometric of the human voice as an input command. The scope of this project is only focused on fan and lamp as an output of home appliances. The proposed system is divided into two main parts which are speech recognition (software development) and electrical home appliance control system (hardware development). In the speech recognition, the process contains two main modules which are feature extraction and pattern matching. Meanwhile, in hardware development, the microcontroller board is used to process the home appliance system such that the system can be operated automatically. It is expected that this project allows the user to conveniently control the home appliances by using voice. The experiment shows that the highest performance of the system achieves an accuracy of 83.25% with the number of k, training, and testing is 1, 20, and 40 respectively.

A modified active disturbance rejection control for fast steering mirror in aerospace application

Active disturbance rejection control (ADRC) has proved to be an effective tool in dealing with engineering problems related to the plant along with dynamic uncertainties, disturbances, nonlinearities. A modified active disturbance rejection control (MADRC) method is proposed to design the Fast Steering Mirror (FSM) controller. The modified extended state observer (ESO) incorporate with identified plant information is employed to reject the disturbances, nonlinear factors and modelling errors which meant to constraint the plant to a nominal form. Therewith, an well-designed analytical method is introduced to calculate the parameters of a feedback loop to force the plant response like a pre-designed model. Besides, a Lead-Leg Controller (LLC) consisting of two Tracking-Differentiator (TD) is designed to improve the performance of the controller. LLC generates a filtered output based on the expected plant dynamics to increase the FSM bandwidth, and LLC can also help to improve the stability of the system by restricting the tracking error in a limited range when there is an abrupt command input. The MADRC strategy is proven an effective solution for FSM. Simulation results show that the MADRC method can achieve a excellent performance without tedious tuning work. The experiment results proved the validation in practical application.

Designing and Evaluating Hand-to-Hand Gestures with Dual Commodity Wrist-Worn Devices

Hand gestures provide a natural and easy-to-use way to input commands. However, few works have studied the design space of bimanual hand gestures or attempted to infer gestures that involve devices on both hands. We explore the design space of hand-to-hand gestures, a group of gestures that are performed by touching one hand with the other hand. Hand-to-hand gestures are easy to perform and provide haptic feedback on both hands. Moreover, hand-to-hand gestures generate simultaneous vibration on two hands that can be sensed by dual off-the-shelf wrist-worn devices. In this work, we derive a hand-to-hand gesture vocabulary with subjective ratings from users and select gesture sets for real-life scenarios. We also take advantage of devices on both wrists to demonstrate their gesture-sensing capability. Our results show that the recognition accuracy for fourteen gestures is 94.6% when the user is stationary, and the accuracy for five gestures is 98.4% or 96.3% when the user is walking or running, respectively. This is significantly more accurate than a single device worn on either wrist. Our further evaluation also validates that users can easily remember hand-to-hand gestures and use our technique to invoke commands in real-life contexts.

Operational Feasibility Analysis of the Multimodal Controller Working Position “TriControl”

Current Air Traffic Controller working positions (CWPs) are reaching their capacity owing to increasing levels of air traffic. The multimodal CWP prototype TriControl combines automatic speech recognition, multitouch gestures, and eye-tracking, aiming for more natural and improved human interaction with air traffic control systems. However, the prototype has not yet undergone systematic evaluation with respect to feasibility. This paper evaluates the operational feasibility, focusing on the system usability of the approach CWP TriControl and its fulfillment of operational requirements. Fourteen controllers took part in a simulation study to evaluate the TriControl concept. The active approach controllers among the group of participants served as the main core target subgroup. The ratings of all controllers in the TriControl assessment were, on average, generally in slight agreement, with just a few showing statistical significance. However, the active approach controllers performed better and rated the system much more positively. The active approach controllers were strongly positive regarding the system usability and acceptance of this early-stage prototype. Particularly, ease of use, user-friendliness, and learnability were perceived very positively. Overall, they were also satisfied with the command input procedure, and would use it for their daily work. Thus, the participating controllers encourage further enhancements to be made to TriControl.

The Manipulation of Real-Time Kinect-Based Robotic Arm Using Double-Hand Gestures

This study focused on utilizing the Kinect depth sensor to track double-hand gestures and control a real-time robotic arm. The control system is mainly composed of the microprocessor, a color camera, the depth sensor, and the robotic arm. The Kinect depth sensor was used to take photos of the human body to analyze the skeleton of a human body and obtain the relevant information. Such information was used to identify the gestures of the left hand and the left palm of the user. The gesture of left hand was used as an input command device. The gesture of the right hand was used for imitation movement teaching of robotic arm. From the depth sensor, the real-time images of the human body and the deep information of each joint were collected and converted to the relative positions of the robotic arm. Combining forward kinematics and inverse kinematics and D-H link, the gesture information of the right hand was calculated, which was converted via coordinates into each angle of the motor of the robotic arm. From the color camera, when the left palm was not detected, the user could simply use the right hand to control the action and movement of the real-time robotic arm. When the left palm was detected and 5 fingertips were identified, it meant the start of recording the real-time imitation movement of the robotic arm by the right hand. When 0 fingertip was identified, it meant the stoppage of the above recording. When 2 fingertips were identified, the user could not only control the real-time robotic arm but also repeat the recorded actions.

Optimal preview control design of pneumatic servo system: a comparative analysis

This paper describes the development of a preview controller based on optimal control theory for positioning control of a pneumatic servo system. The preview theory of tracking control using the future information has been studied extensively but did not get enough applications in the field of servo control systems, used in modern industrial processing, that will increase the accuracy and efficiency of the industrial product. In the pneumatic servo system, the position of pneumatic cylinder is the command input and its future information can be known a priori. The proposed control system is composed of feedback from the state of the plant and feed-forward from the reference signal by using future information. The validity of the proposed control system is confirmed by simulation results and comparative analysis is done with standard optimal control. Simulation results imply that good performance and good phase characteristics can be obtained using optimal preview controller.

Ensemble Gaussian mixture model-based special voice command cognitive computing intelligent system

Dysarthria is a speech disorder caused by stroke, Parkinson’s disease, neurological injury, or tumors that damage the nervous system and weaken the speech quality. Developing a unique voice command system for Dysarthric speech helps to recognize impaired speech and convert them into text or input commands. Hidden Markov Model (HMM) is one of the widely used generative model-based classifiers for Dysarthric speech recognition. But due to insufficient training data, HMM doesn’t provide optimal results on overlapping classes. We propose an ensemble Gaussian mixture model to recognize impaired speech more accurately. Our model converts the sequence of feature vectors into a fixed dimensional representation of patterns with varying lengths. The performance efficiency of the proposed model is evaluated on the Dysarthric UA-speech benchmark dataset. The discriminatory information provided by the proposed approach yields better classification accuracy even for shallow intelligibility words compared to conventional HMM.

A Cascaded Multimodal Natural User Interface to Reduce Driver Distraction

Natural user interfaces (NUI) have been used to reduce driver distraction while using in-vehicle infotainment systems (IVIS), and multimodal interfaces have been applied to compensate for the shortcomings of a single modality in NUIs. These multimodal NUIs have variable effects on different types of driver distraction and on different stages of drivers' secondary tasks. However, current studies provide a limited understanding of NUIs. The design of multimodal NUIs is typically based on evaluation of the strengths of a single modality. Furthermore, studies of multimodal NUIs are not based on equivalent comparison conditions. To address this gap, we compared five single modalities commonly used for NUIs (touch, mid-air gesture, speech, gaze, and physical buttons located in a steering wheel) during a lane change task (LCT) to provide a more holistic view of driver distraction. Our findings suggest that the best approach is a combined cascaded multimodal interface that accounts for the characteristics of a single modality. We compared several combinations of cascaded multimodalities by considering the characteristics of each modality in the sequential phase of the command input process. Our results show that the combinations speech + button, speech + touch, and gaze + button represent the best cascaded multimodal interfaces to reduce driver distraction for IVIS.

Haptic Teleoperation of UAVs Through Control Barrier Functions

This article presents a novel approach to haptic teleoperation. Specifically, we use control barrier functions (CBFs) to generate force feedback to help human operators safely fly quadrotor UAVs. CBFs take a control signal as input and output a control signal that is as close as possible to the initial control signal, while also meeting specified safety constraints. In the proposed method, we generate haptic force feedback based on the difference between a command issued by the human operator and the safe command returned by a CBF. In this way, if the user issues an unsafe control command, the haptic feedback will help guide the user towards the safe input command that is closest to their current command. We conducted a within-subject user study, in which 12 participants flew a simulated UAV in a virtual hallway environment. Participants completed the task with our proposed CBF-based haptic feedback, no haptic feedback, and haptic feedback generated via parametric risk fields, which is a state-of-the-art method described in the literature. The results of this study show that CBF-based haptic feedback can improve a human operator's ability to safely fly a UAV and reduce the operator's perceived workload, without sacrificing task efficiency.

Haptic Interface Device Using Cable Tension Based on Ultrasonic Phased Array

This paper presents a new haptics interface device based on cable-driven parallel manipulators and an ultrasonic phased array. Unlike previous studies, our method combines force and tactile feedback at the same time. It consists of two main parts: force feedback generated by cable tension and tactile feedback provided by an ultrasonic phased array. While these parts have no direct interference, combining them can offer a synergistic effect on haptic perception. Through a series of experiments, the tactile rendering algorithm for this device is established. The output pressure of ultrasonic phased array is dependent on input command value, modulation frequency, modulation waveform, and the position of the focal point in the workspace. The results of psychophysical experiments are evaluated to determine the absolute threshold of perceivable ultrasonic tactile feedback when the cable-driven force acts on fingers. Finally, we carry out a test to confirm the accuracy advantage of our system in the virtual environment. The results of our study indicate that this device has a wide range of applications in the field of aerial haptic display.

Perceptually Correct Haptic Rendering in Mid-Air Using Ultrasound Phased Array

This paper provides a perceptually transparent rendering algorithm for an ultrasound-based mid-air haptic device. In a series of experiments, we derive a systematic mapping function relating from the device command value to final user's perceived magnitude of a mid-air vibration feedback. The algorithm is designed for the ultrasonic mid-air haptic interface that is capable of displaying vibro-tactile feedback at a certain focal point in mid-air through ultrasound phased array technique. The perceived magnitude at the focal point is dependent on input parameters, such as input command intensity, modulation frequency, and position of the focal point in the work-space. This algorithm automatically tunes these parameters to ensure that the desired perceived output at the user's hand is precisely controlled. Through a series of experiments, the effect of the aforementioned parameters on the physical output pressure are mapped, and the effect of this output pressure to the final perceived magnitude is formulated, resulting in the mapping from the different parameters to the perceived magnitude. Finally, the overall transparent rendering algorithm was evaluated, showing better perceptual quality than rendering with simple intensity command.

Development of trajectory and force controllers for 3-joint string-tube actuated finger prosthesis based on bond graph modeling

The bond graph modeling and simulation of a redundant underactuated three-joint string-tube based prosthetic finger mechanism in the unlike configuration, for a class of prosthesis for a partially impaired hand, was presented for the implementation of the concept of opposition space for prehensile tasks. This work is its extension and systematically develops the structural basis and models for trajectory and force control schemes. The models emulate the abstract virtual domain of the intended task and the Central Nervous System (CNS), which provides control commands to the natural active finger joints, for the tracking control of desired position and force trajectories. The model for force control tracking has been developed systematically by utilizing the concept of causality from the bond graph model, where the motion of an infinitesimal mass is used as a trajectory input command for the generation of the desired force at the fingertip-object interface. The control models derived from their respective bond graphs do not require inverse kinematics to be worked out, and have been validated through simulations.

Adaptive Kinematic Mapping Based on Chebyshev Interpolation: Application to Flexure-Jointed Micromanipulator Control

For flexure-jointed mechanisms and manipulators, accurate kinematic models typically involve complex nonlinear descriptions of elastic behavior and are ill-suited for real-time control computations. An alternative approach is to use multidimensional function approximation methods, with optimization via online kinematic identification and calibration. In this article, a function interpolation approach is proposed based on Chebyshev approximation theory for near-optimal minimization of maximum positioning errors. The method allows fast calibration procedures using a small number of data points. The implementation involves a correction mapping that operates on command input variables before an approximate inverse kinematic model is applied. An adaptation algorithm is further proposed that can be used to update and refine mappings: 1) in a localized space for improved precision for the current task, or 2) globally by using calibration points chosen to match the Chebyshev nodes of the overall workspace. Results are shown for simulation of a flexure-jointed X–Y motion stage and for experiments on a X–Y–Z micromanipulation platform with Delta-type parallel kinematics. For the experiments, direct measurement of platform position was achieved using a microscope vision system. The proposed method gave order-of-magnitude improvements in positioning accuracy compared with the pseudo-rigid-body modeling approach and was found to out-perform direct visual servoing when operating with similar image capture rates.

Helicopter stability and control derivatives identification in different flight conditions

Helicopter flight simulators require high-fidelity dynamic models based on a set of stability and control derivatives (SCDs). For different flight conditions, specific sets of SCDs must be identified and utilized in the flight simulator software. In this study, nine sets of SCDs of the AS 355 F2 helicopter have been estimated from experimental flight-testing data, considering different combinations of velocity, mass, and altitude. Linear dynamic models with eight states and four controls were determined for the different flight conditions, using the output-error method. The associated optimization problem was solved with the Gauss–Newton algorithm, and identification reliability was evaluated by the Cramer–Rao (CR) and relative correlation coefficient (RCC) indices. An instrumented AS 355 F2 Helicopter belonging to the Instituto de Pesquisas e Ensaios em Voo, Departamento de Ciencia e Tecnologia Aeroespacial, Sao Jose dos Campos, Brazil, was utilized for data collection in different flight conditions, totaling 16 test hours. The following natural dynamic modes have been excited: phugoid, short period, spiral and Dutch roll, by command inputs of the type: frequency sweep (sinusoidal), doublet pulse and 3-2-1-1. Considering CR and RCC criteria, the most appropriate maneuver for exciting each dynamic mode is suggested, and the sets of the nine flight conditions are SCDs reported.

Steady State Frequency Response Design of Finite Time Iterative Learning Control

Steady state frequency response is a major design approach in control theory, and is normally used to obtain reasonable response to commands up to a chosen or obtainable bandwidth. By contrast, iterative learning control (ILC) aims to converge to zero error at all time steps of a finite time tracking task, using an iterative update of the command to a control system. It asks for zero error during the transient as well as steady state response phases. This paper develops a rigorous design approach to apply frequency response to the ILC problem. The gains in an FIR compensator from repetitive control are used to fill the ILC learning gain matrix. The compensator mimics the inverse frequency response. This paper does not ask for zero error for one or more initial times steps, in oder to address a well posed problem that is not trying to converge to an unstable command input. It is seen that by adjusting just one gain in the upper left corner of the learning matrix, the ILC can be stabilized in spite of the length of time involving transients. This makes a very simple design process. One can also adjust two gains in the first row, or better a two-by-two block, but further adjustment seems of limited benefit. The frequency based ILC design developed here is compared to existing ILC design methods, and shown to be vastly superior in convergence rate. It also has the advantage that the ILC can be designed directly from frequency response test data without creating a mathematical model.

Designing User Interaction using Gesture and Speech for Mixed Reality Interface

Mixed Reality (MR) is the next evolution of humans interacting with computer as MR can combine the physical environment and digital environment and making them coexist with each other [1]. Interaction is still a valid research area in MR, and this paper focuses on interaction rather than other research areas such as tracking, calibration, and display [2] because the current interaction technique still not intuitive enough to let the user interact with the computer. This paper explores the user interaction using gesture and speech interaction for 3D object manipulation in mixed reality environment. The paper explains the design stage that involves interaction using gesture and speech inputs to enhance user experience in MR workspace. After acquiring gesture input and speech commands, MR prototype is proposed to integrate the interaction technique using gesture and speech. The paper concludes with results and discussion.

Process Monitoring Dataset from the Additive Manufacturing Metrology Testbed (AMMT): "Three-Dimensional Scan Strategies".

This document provides details on the files available in the dataset "20180708-HY-3D Scan Strategies" pertaining to a 3D additive manufacturing build performed on the Additive Manufacturing Metrology Testbed (AMMT)by Ho Yeung on July 8, 2018. The files include the input command files and in-situ process monitoring data, and metadata. This data is the first of future planned "AMMT Process Monitoring Reference Datasets," as part of the Metrology for Real-Time Monitoring of Additive Manufacturing project.

Optimal preview control design of pneumatic servo system: a comparative analysis

This paper describes the development of a preview controller based on optimal control theory for positioning control of a pneumatic servo system. The preview theory of tracking control using the future information has been studied extensively but did not get enough applications in the field of servo control systems, used in modern industrial processing, that will increase the accuracy and efficiency of the industrial product. In the pneumatic servo system, the position of pneumatic cylinder is the command input and its future information can be known a priori. The proposed control system is composed of feedback from the state of the plant and feed-forward from the reference signal by using future information. The validity of the proposed control system is confirmed by simulation results and comparative analysis is done with standard optimal control. Simulation results imply that good performance and good phase characteristics can be obtained using optimal preview controller.