Human Intention Detection Research Articles

Modeling and Analysis of Human Comfort in Human-Robot Collaboration.

The emergence and recent development of collaborative robots have introduced a safer and more efficient human-robot collaboration (HRC) manufacturing environment. Since the release of COBOTs, a great amount of research efforts have been focused on improving robot working efficiency, user safety, human intention detection, etc., while one significant factor-human comfort-has been frequently ignored. The comfort factor is critical to COBOT users due to its great impact on user acceptance. In previous studies, there is a lack of a mathematical-model-based approach to quantitatively describe and predict human comfort in HRC scenarios. Also, few studies have discussed the cases when multiple comfort factors take effect simultaneously. In this study, a multi-linear-regression-based general human comfort prediction model is proposed under human-robot collaboration scenarios, which is able to accurately predict the comfort levels of humans in multi-factor situations. The proposed method in this paper tackled these two gaps at the same time and also demonstrated the effectiveness of the approach with its high prediction accuracy. The overall average accuracy among all participants is 81.33%, while the overall maximum value is 88.94%, and the overall minimum value is 72.53%. The model uses subjective comfort rating feedback from human subjects as training and testing data. Experiments have been implemented, and the final results proved the effectiveness of the proposed approach in identifying human comfort levels in HRC.

Task-oriented human-robot interaction control of a robotic glove utilizing forearm electromyography

Task-oriented myoelectric assistive control of a robotic glove based on forearm electromyography for practical grasping tasks is investigated in this work. Three grasping functions are considered: human intent detection of grasping tasks, grasping mode recognition, and grasping force control. Different combinations of forearm electromyographic signals are adopted for the three functions. Firstly, the overall electromyographic signal is used to trigger the whole grasping task. Secondly, a novel Long Short-Term Memory network is utilized to classify various grasping modes, including pinch grasping and palmar grasping, by analyzing eight-channel electromyographic signals. Thirdly, two-channel proportional control of grasping force using electromyographic flexor/extensor signals is adopted for the robotic glove such that the human hand can relax during the grasping task, while the robotic glove maintains the grasping force. To this end, finite state machine based hierarchical control architecture is proposed for the whole grasping task. Experiments are conducted to validate the proposed task-oriented assistive control method, and the results clearly demonstrates the potential of the proposed method in rehabilitation therapy.

Machine Learning for Human Motion Intention Detection.

The gait pattern of exoskeleton control conflicting with the human operator's (the pilot) intention may cause awkward maneuvering or even injury. Therefore, it has been the focus of many studies to help decide the proper gait operation. However, the timing for the recognization plays a crucial role in the operation. The delayed detection of the pilot's intent can be equally undesirable to the exoskeleton operation. Instead of recognizing the motion, this study examines the possibility of identifying the transition between gaits to achieve in-time detection. This study used the data from IMU sensors for future mobile applications. Furthermore, we tested using two machine learning networks: a linearfFeedforward neural network and a long short-term memory network. The gait data are from five subjects for training and testing. The study results show that: 1. The network can successfully separate the transition period from the motion periods. 2. The detection of gait change from walking to sitting can be as fast as 0.17 s, which is adequate for future control applications. However, detecting the transition from standing to walking can take as long as 1.2 s. 3. This study also find that the network trained for one person can also detect movement changes for different persons without deteriorating the performance.

Lower Limb Exoskeleton for Rehabilitation with Flexible Joints and Movement Routines Commanded by Electromyography and Baropodometry Sensors.

This paper presents the development of an instrumented exoskeleton with baropodometry, electromyography, and torque sensors. The six degrees of freedom (Dof) exoskeleton has a human intention detection system based on a classifier of electromyographic signals coming from four sensors placed in the muscles of the lower extremity together with baropodometric signals from four resistive load sensors placed at the front and rear parts of both feet. In addition, the exoskeleton is instrumented with four flexible actuators coupled with torque sensors. The main objective of the paper was the development of a lower limb therapy exoskeleton, articulated at hip and knees to allow the performance of three types of motion depending on the detected user's intention: sitting to standing, standing to sitting, and standing to walking. In addition, the paper presents the development of a dynamical model and the implementation of a feedback control in the exoskeleton.

A Control Framework for Adaptation of Training Task and Robotic Assistance for Promoting Motor Learning With an Upper Limb Rehabilitation Robot

Robot-assisted rehabilitation has been a promising solution to improve motor learning of neurologically impaired patients. State-of-the-art control strategies are typically limited to the ignorance of heterogeneous motor capabilities of poststroke patients and therefore intervene suboptimally. In this article, we propose a control framework for robot-assisted motor learning, emphasizing the detection of human intention, generation of reference trajectories, and modification of robotic assistance. A real-time trajectory generation algorithm is presented to extract the high-level features in active arm movements using an adaptive frequency oscillator (AFO) and then integrate the movement rhythm with the minimum-jerk principle to generate an optimal reference trajectory, which synchronizes with the motion intention in the patient as well as the motion pattern in healthy humans. In addition, a subject-adaptive assistance modification algorithm is presented to model the patient’s residual motor capabilities employing spatially dependent radial basis function (RBF) networks and then combining the RBF-based feedforward controller with the impedance feedback controller to provide only necessary assistance while simultaneously regulating the maximum-tolerated error during trajectory tracking tasks. We conduct simulation and experimental studies based on an upper limb rehabilitation robot to evaluate the overall performance of the motor-learning framework. A series of results showed that the difficulty level of reference trajectories was modulated to meet the requirements of subjects’ intended motion, furthermore, the robotic assistance was compliantly optimized in response to the changing performance of subjects’ motor abilities, highlighting the potential of adopting our framework into clinical application to promote patient-led motor learning.

Electroencephalogram and surface electromyogram fusion-based precise detection of lower limb voluntary movement using convolution neural network-long short-term memory model.

The electroencephalogram (EEG) and surface electromyogram (sEMG) fusion has been widely used in the detection of human movement intention for human–robot interaction, but the internal relationship of EEG and sEMG signals is not clear, so their fusion still has some shortcomings. A precise fusion method of EEG and sEMG using the CNN-LSTM model was investigated to detect lower limb voluntary movement in this study. At first, the EEG and sEMG signal processing of each stage was analyzed so that the response time difference between EEG and sEMG can be estimated to detect lower limb voluntary movement, and it can be calculated by the symbolic transfer entropy. Second, the data fusion and feature of EEG and sEMG were both used for obtaining a data matrix of the model, and a hybrid CNN-LSTM model was established for the EEG and sEMG-based decoding model of lower limb voluntary movement so that the estimated value of time difference was about 24 ∼ 26 ms, and the calculated value was between 25 and 45 ms. Finally, the offline experimental results showed that the accuracy of data fusion was significantly higher than feature fusion-based accuracy in 5-fold cross-validation, and the average accuracy of EEG and sEMG data fusion was more than 95%; the improved average accuracy for eliminating the response time difference between EEG and sEMG was about 0.7 ± 0.26% in data fusion. In the meantime, the online average accuracy of data fusion-based CNN-LSTM was more than 87% in all subjects. These results demonstrated that the time difference had an influence on the EEG and sEMG fusion to detect lower limb voluntary movement, and the proposed CNN-LSTM model can achieve high performance. This work provides a stable and reliable basis for human–robot interaction of the lower limb exoskeleton.

Improved Motion Classification With an Integrated Multimodal Exoskeleton Interface.

Human motion intention detection is an essential part of the control of upper-body exoskeletons. While surface electromyography (sEMG)-based systems may be able to provide anticipatory control, they typically require exact placement of the electrodes on the muscle bodies which limits the practical use and donning of the technology. In this study, we propose a novel physical interface for exoskeletons with integrated sEMG- and pressure sensors. The sensors are 3D-printed with flexible, conductive materials and allow multi-modal information to be obtained during operation. A K-Nearest Neighbours classifier is implemented in an off-line manner to detect reaching movements and lifting tasks that represent daily activities of industrial workers. The performance of the classifier is validated through repeated experiments and compared to a unimodal EMG-based classifier. The results indicate that excellent prediction performance can be obtained, even with a minimal amount of sEMG electrodes and without specific placement of the electrode.

Satisfying Task Completion and Assist-as-Needed Performance in Robotic Exoskeletons

Assist-as-needed (AAN) control approaches aim to minimize the intervention of robots in robot-aided rehabilitation therapy when patients are able to perform the task. The existing approaches generally contain a forgiving controller which fails to satisfactorily complete the task when patients stay passive or apply resistive/opposing forces. Task completion in such situations is important since it efficiently trains and guides users on how to perform the exercise during active participation. In this study, we have developed an AAN control scheme that provides significant freedom to users when they can perform the exercise while it provides sufficient support for the precise completion of the task when users are not able to do so. These features have been achieved by modulating the impedance of the robot and designing the controller in the velocity domain. A new velocity tracking controller is proposed which does not utilize velocity measurements, and two methods are considered to measure human inputs: force sensor-based method and disturbance observer-based method. In the first approach, the controller does not require the dynamics of the human-robot system, and in the second approach, partial information of the dynamics is required. The performance of the controller is evaluated on an upper-limb robotic exoskeleton. Note to Practitioners-The classic control methods utilized in rehabilitation robots for the active participation of patients are usually low-gain controllers that allow patients to deviate from the desired motion path. However, this approach results in poor task completion when users are not able to do the task. This paper proposes a control scheme for robotic exoskeletons that simultaneously satisfies the two objectives, namely, the low intervention of the robot when users can perform the task, and precise task completion when users cannot handle it. The proposed controller does not require velocity measurements and the precise dynamic model of the system. The approach needs human intention detection, which can be achieved by using force sensors or a disturbance observer. While the second method reduces the complexity and cost of the mechanical parts, it requires the dynamic model of the robot. The tests conducted on an upper-limb robotic exoskeleton show the satisfactory performance of both methods.

Effective Multi-Mode Grasping Assistance Control of a Soft Hand Exoskeleton Using Force Myography.

Human intention detection is fundamental to the control of robotic devices in order to assist humans according to their needs. This paper presents a novel approach for detecting hand motion intention, i.e., rest, open, close, and grasp, and grasping force estimation using force myography (FMG). The output is further used to control a soft hand exoskeleton called an SEM Glove. In this method, two sensor bands constructed using force sensing resistor (FSR) sensors are utilized to detect hand motion states and muscle activities. Upon placing both bands on an arm, the sensors can measure normal forces caused by muscle contraction/relaxation. Afterwards, the sensor data is processed, and hand motions are identified through a threshold-based classification method. The developed method has been tested on human subjects for object-grasping tasks. The results show that the developed method can detect hand motions accurately and to provide assistance w.r.t to the task requirement.

Making more sense of human intention detection via reconstruction models

Movement intention recognition paves the path to developing Brain-Computer Interface (BCI) applications. Current research mostly focuses on questions like "which hand is intended to move". While answering questions like "whether a hand is intended to move" is more desirable for widely realworld applications, because we cannot continuously perform intention tasks during usage, and the gap periods may cause unintended operation resulting in system failures. However, this kind of intention detection task is more difficult, since for a "whether" problem, it is hard to know what the "not" situation is and consequently to acquire training samples for the "not" situation. Furthermore, the occurrence of genuine intentions is comparatively scarce and unexpected, making the intention detection task hard and computation-consuming. To tackle this problem, we propose a Reconstruction-based Intention Detection (RID) framework, which utilises a reconstruction model to represent a high-level abstraction of EEG signals and leverages the reconstruction errors to determine "whether" there is a movement intention. Our framework is not only theoretically flexible and robust to any sophisticated real-world scenarios but also hand-crafted feature and domain knowledge free. Comprehensive experiments on detecting movement intention tasks with different reconstruction models demonstrate the promising performance of the proposed reconstruction intention detection framework.

Hybrid Active Control With Human Intention Detection of an Upper-Limb Cable-Driven Rehabilitation Robot

Rehabilitation robots play an increasingly important role in the recovery of motor function for stroke. To ensure a natural physical human-robot interaction (pHRI) and enhance the active participation of subjects, it is necessary for the robots to understand the human intention and cooperate actively with humanlike characteristics. This study proposed a hybrid active control algorithm with human motion intention detection. The motion intention was defined as the desired position and velocity, which were continuously estimated according to the human upper-limb model and minimum jerk model, respectively. The motion intention was then fed into a hybrid force and position controller of an upper-limb cable driven rehabilitation robot (CDRR). And a three-dimensional reaching task without predefined trajectory was employed to validate the effectiveness of the proposed control algorithm. Experimental results showed that the control algorithm could continuously recognize the human motion intention and enabled the robot better movement performance indicated as smaller offset error, smoother trajectory, and lower impact. The proposed method could guarantee a natural pHRI and improve the engagement of the subjects, which has great potential in clinical applications.

Design of a Capacitance Sensor for Human Intention Detection of Daily Living Activities

As many countries enter an aging society, the demand of wearable robots are increasing. To apply wearable robots in daily life, the intention recognition of the wearers is of great important. Although this research field has been extensively studied in the last decade, still the physical intention recognition based approaches are facing different challenges and in particular when used in assistive and rehabilitation scenarios. In this paper, we propose to monitor muscular activities of the shank and movement of the ankle joint by using capacitance sensors. The proposed capacitance sensor monitor the electrodermal activities under the human skin. The human skin has capacitance when an external current is applied and muscular activities can change the capacitance. To verify the proposed capacitance sensor, the human joint angle was measured by two inertial sensors and the capacitance sensor on the shank at once. The electromyogram(EMG) signals were also compared to verify the performance of the intention detection device based on the capacitance sensors.

Sensorless force estimation of end-effect upper limb rehabilitation robot system with friction compensation

In human robot interaction systems, human intent detection plays an important role to improve the interactive performances and then the rehabilitation effects. A study is proposed to estimate the interactive forces that indirectly detect the human motion intent. A disturbance observer is designed to estimate interactive torques and friction forces without force sensors, and then a friction force model is constructed to estimate the friction force in the robot system. To detect the human–robot interaction force, we subtract the friction force from disturbance observer estimation result. Several experiments were performed to test the performances of the proposed methods. Those methods were applied in an end-effect upper limb rehabilitation robot system. The results show that the precision of the estimated sensor force can increase 5% than the force sensor. The senseless force estimation method we proposed in this article can be an alternative option in force control tasks when force sensors are not suitable.

Ankle Dorsiflexion Strength Monitoring by Combining Sonomyography and Electromyography.

Ankle dorsiflexion produced by Tibialis Anterior (TA) muscle contraction plays a significant role during human walking and standing balance. The weakened function or dysfunction of the TA muscle often impedes activities of daily living (ADL). Powered ankle exoskeleton is a prevalent technique to treat this pathology, and its intelligent and effective behaviors depend on human intention detection. A TA muscle contraction strength monitor is proposed to evaluate the weakness of the ankle dorsiflexion. The new method combines surface electromyography (sEMG) signals and sonomyography signals to estimate ankle torque during a voluntary isometric ankle dorsiflexion. Changes in the pennation angle (PA) are derived from the sonomyography signals. The results demonstrate strong correlations among the sonomyography-derived PA, the sEMG signal, and the measured TA muscle contraction force. Especially, the TA muscle strength monitor approximates the TA muscle strength measurement via a weighted summation of the sEMG signal and the PA signal. The new method shows an improved linear correlation with the muscle strength, compared to the correlations between the muscle strength and sole sEMG signal or sole PA signal, where the R-squared values are improved by 4.21 % and 1.99 %, respectively.

Transferrable Expertise From Bionic Arms to Robotic Exoskeletons: Perspectives for Stroke and Duchenne Muscular Dystrophy

Upper extremity function is affected by a variety of neurological conditions. Robotic exoskeletons offer a potential solution for motor restoration. However, their systematic adoption is limited by challenges relative to human intention detection and device control. This position article offers a focused perspective on this topic. That is, on how knowledge gained from the design and implementation of human–machine interfaces (HMIs) for bionic arms can benefit the field of rehabilitation exoskeletons. Three broadly used HMIs in bionic arms are here investigated, including surface electromyography, impedance, and body-powered control. We propose that combinations of these HMIs could push forward upper extremity exoskeleton development. In this context, we provide concrete applicative examples in two selected clinical scenarios, including post-stroke and Duchenne muscular dystrophy individuals. The discussed solutions can open new avenues for the translation of robotic exoskeletons in a large set of clinical settings and enable a class of exoskeleton technologies that could support a broader range of impairment and disease types.

Trajectory adaptation for an impedance controlled cooperative robot according to an operator's force

Abstract The goal of this paper is to design a controller for a cooperative robot based on a human's force that makes the robot's end-effector follow predefined motion primitives. To this aim, based on a 3D model of the robot, the kinematics and dynamics are analyzed at first. Then, an automatic online trajectory generator is designed based on the pre-defined motion primitives and the human operator intention detection. The generated trajectory serves as an input to an impedance controller. The designed impedance controller makes the robot achieve an active compliance for a safe cooperation with the human operator. The proposed controller is implemented on a robot, namely WallMoBot, which is a cooperative robot for heavy glass panels installation. Simulations of the proposed method are presented, and the results show that the method is well suited for the WallMoBot.

Human Intention Detection as a Multiclass Classification Problem: Application in Physical Human–Robot Interaction While Walking

In many physical human–robot interaction scenarios, for successful completion of the tasks, robots should be able to recognize the human partner's intention. One of such scenarios that is studied in this letter is the collaborative task of carrying an object by a human–humanoid pair in which the humanoid should be able to interpret specific human partner's intentions (e.g., start/stop-walking, accelerate, etc.) only through haptic feedback. To address this problem, we first performed human-human experiments and obtained a multiclass classifier (with more than 90% of accuracy) for human intention detection using, as features, arm position relative to the shoulder and interaction forces. The results of the multiclass classification, without any modifications, have been used to develop an interlimb coordinator that was integrated in a modular control architecture into human-robot experiments. The interlimb coordinator receives the sensory data of the upper-body and sends appropriate commands (including start/stop-walking, accelerate, and decelerate commands) to the lower body controller, which is responsible for achieving a stable walking gait. This modular control approach is successfully tested in the human–humanoid experiments with the COMAN robot.

Human intention detection with facial expressions using video analytics

The manuscript should contain an abstract. The abstract should be self-contained and citation-free and should not exceed 200 words. The abstract should state the purpose, approach, results and conclusions of the work.The author should assume that the reader has some knowledge of the subject but has not read the paper. Thus, the abstract should be intelligible and complete in it-self (no numerical references); it should not cite figures, tables, or sections of the paper. The abstract should be written using third person instead of first person.

A review: Motor rehabilitation after stroke with control based on human intent.

Strokes are a leading cause of acquired disability worldwide, and there is a significant need for novel interventions and further research to facilitate functional motor recovery in stroke patients. This article reviews motor rehabilitation methods for stroke survivors with a focus on rehabilitation controlled by human motor intent. The review begins with the neurodevelopmental principles of motor rehabilitation that provide the neuroscientific basis for intuitively controlled rehabilitation, followed by a review of methods allowing human motor intent detection, biofeedback approaches, and quantitative motor rehabilitation assessment. Challenges for future advances in motor rehabilitation after stroke using intuitively controlled approaches are addressed.

Slaves no longer: review on role assignment for human–robot joint motor action

This paper summarizes findings on the growing field of role assignment policies for human–robot motor interaction. This topic has been investigated by researchers in the psychological theory of joint action, in human intention detection, force control, human–human physical interaction, as well as roboticists interested in developing robots with capabilities for efficient motor interaction with humans. Our goal is to promote fruitful interaction between these distinct communities by: ( i) examining the role assignment policies for human–robot joint motor action in experimental psychology and robotics studies; and ( ii) informing researchers in human–human interaction on existing work in the robotic field. After an overview of roles assignment in current robotic assistants, this paper examines key results about shared control between a robot and a human performing interactive motor tasks. Research on motor interaction between two humans has inspired recent developments that may extend the use of robots to applications requiring continuous mechanical interaction with humans.