Intention Recognition Algorithm Research Articles

Vision-and Tactile-Based Continuous Multimodal Intention and Attention Recognition for Safer Physical Human–Robot Interaction

Employing skin-like tactile sensors on robots enhances both the safety and usability of collaborative robots by adding the capability to detect human contact. Unfortunately, simple binary tactile sensors alone cannot determine the context of the human contact—whether it is a deliberate interaction or an unintended collision that requires safety manoeuvres. Many published methods classify discrete interactions using more advanced tactile sensors or by analysing joint torques. Instead, we propose to augment the intention recognition capabilities of simple binary tactile sensors by adding a robot-mounted camera for human posture analysis. Different interaction characteristics, including touch location, human pose, and gaze direction, are used to train a supervised machine learning algorithm to classify whether a touch is intentional or not with an F1-score of 86%. We demonstrate that multimodal intention recognition is significantly more accurate than monomodal analyses with the collaborative robot Baxter. Furthermore, our method can also continuously monitor interactions that fluidly change between intentional or unintentional by gauging the user’s attention through gaze. If a user stops paying attention mid-task, the proposed intention and attention recognition algorithm can activate safety features to prevent unsafe interactions. We also employ a feature reduction technique that reduces the number of inputs to five to achieve a more generalized low-dimensional classifier. This simplification both reduces the amount of training data required and improves real-world classification accuracy. It also renders the method potentially agnostic to the robot and touch sensor architectures while achieving a high degree of task adaptability. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Whenever a user interacts physically with a robot, such as in collaborative manufacturing, the robot may respond to unintended touch inputs from the user. This may be through body collisions or that the user is suddenly distracted and is no longer paying attention to what they are doing. We propose an easy-to-implement method to augment safety of physical human-robot collaboration by determining whether the touch from a user is intentional or not through the use of robot-mounted basic touch sensors and computer vision. The algorithm examines the location of the user’s hands relative to the touched sensors in addition to observing where the user is looking. Machine learning is then used to classify in real-time, with an F1-score of 86%, whether a touch is intentional or not such that the robot can react accordingly. The method is particularly applicable in collaborative manufacturing contexts, but can also be applied anywhere where a user physically interacts with a robot. We demonstrate the utility of the method in enhancing safety during human-robot collaboration through a simulated collaborative manufacturing scenario with the robot Baxter, but the method can easily be adapted to other systems.

Research on Improved Dual Channel Medical Short Text Intention Recognition Algorithm

Abstract The increasing application of medical robots in the healthcare sector underscores the critical importance of intent recognition in enhancing the interaction and assistance capabilities of these robots. Traditional intent recognition methods utilize convolutional neural networks (CNNs) for text analysis but often fall short in capturing global features, resulting in incomplete information. To address this challenge, this paper introduces an innovative approach by combining an enhanced CNN with bidirectional gated recurrent units (BiGRU) to construct a dual-channel short-text intent recognition model. This model effectively leverages both local and global features to more accurately comprehend user needs and intentions. Experimental results demonstrate that this model excels, achieving an accuracy rate of 96.68% and an F1 score of 96.67% on the THUCNews_Title dataset. In comparison to conventional intent recognition models, it exhibits significantly improved performance, thereby providing substantial support for medical robots in patient care and assisting healthcare professionals.

Recognition and Prediction of Pedestrian Hazardous Crossing Intentions in Visual Field Obstruction Areas Based on IPVO-LSTM

Pedestrians who suddenly cross the street from within the blind spot of a vehicle’s field of view can pose a significant threat to traffic safety. The dangerous pedestrian crossing intentions in view-obscured scenarios have not received as much attention as the prediction of pedestrian crossing intentions. In this paper, we present a method for recognizing and predicting the dangerous crossing intention of pedestrians in a view-obscured region based on the interference, pose, velocity observation–long short-term memory (IPVO-LSTM) algorithm from a road-based view. In the first step, the road-based camera captures the pedestrian’s image. Then, we construct a pedestrian interference state feature module, pedestrian three-dimensional pose feature module, pedestrian velocity feature module, and pedestrian blind observation state feature module and extract the corresponding features of the studied pedestrians. Finally, the pedestrian hazard crossing intention prediction module based on a feature-fused LSTM (ff-LSTM) and attention mechanism is used to fuse and process the above features in a cell state process to recognize and predict the pedestrian hazard crossing intention in the blind visual area. Experiments are compared with current common algorithms in terms of the input parameter selection, intention recognition algorithm, and intention prediction time range, and the experimental results validate our state-of-the-art method.

Design and Control of a Lower Limb Rehabilitation Robot Based on Human Motion Intention Recognition with Multi-Source Sensor Information

The research on rehabilitation robots is gradually moving toward combining human intention recognition with control strategies to stimulate user involvement. In order to enhance the interactive performance between the robot and the human body, we propose a machine-learning-based human motion intention recognition algorithm using sensor information such as force, displacement and wheel speed. The proposed system uses the bi-directional long short-term memory (BILSTM) algorithm to recognize actions such as falling, walking, and turning, of which the accuracy rate has reached 99.61%. In addition, a radial basis function neural network adaptive sliding mode controller (RBFNNASMC) is proposed to track and control the patient’s behavioral intention and the gait of the lower limb exoskeleton and to adjust the weights of the RBF network using the adaptive law. This can achieve a dynamic estimation of the human–robot interaction forces and external disturbances, and it gives the exoskeleton joint motor a suitable driving torque. The stability of the controller is demonstrated using the Lyapunov stability theory. Finally, the experimental results demonstrate that the BILSTM classifier has more accurate recognition than the conventional classifier, and the real-time performance can meet the demand of the control cycle. Meanwhile, the RBFNNASMC controller has a better gait tracking effect compared with the PID controller.

Combined AGADESN with DBSCAN Algorithm for Cluster Target Motion Intention Recognition

In this paper, we consider the problem of motion intention recognition for cluster targets with splitting behaviour and lack of motion prior information. This is a challenge to the classical Bayesian inference based intention recognition algorithms because they rely heavily on a priori knowledge. In order to solve these problems, a joint algorithm of deep echo state network optimized by adaptive genetic algorithm (AGADESN) and DBSCAN clustering algorithm is proposed in this paper. We use improved Olfati-Saber model with direction noise to generate cluster motion and use the cluster motion data to drive AGADESN algorithm to predict cluster destination, which achieves higher destination prediction accuracy than DESN algorithm. We innovatively design the motion similarity distance (MSD) and take the destination prediction output as one of the distance inputs, alleviating the lack of differentiation among different cluster targets caused by only relying on speed and position distance at the early stage of cluster motion. Based on the MSD, DBSCAN clustering algorithm is used to identify clusters in the field of view to determine whether splitting behaviour occurs. Simulation results demonstrate the effectiveness of the proposed algorithm in cluster target motion intention recognition and its superiority over DESN algorithm and DBSCAN algorithm only based on speed and position distance.

Intention Prediction of a Hypersonic Glide Vehicle Using a Satellite Constellation Based on Deep Learning

Tracking of hypersonic glide vehicles (HGVs) by a constellation tracking and observation system is an important part of the space-based early warning system. The uncertainty in the maneuver intentions of HGVs has a non-negligible impact on the tracking and observation process. The cooperative scheduling of multiple satellites in an environment of uncertainty in the maneuver intentions of HGVs is the main problem researched in this paper. For this problem, a satellite constellation tracking decision method that considers the HGVs’ maneuver intentions is proposed. This method is based on building an HGV maneuver intention model, developing a maneuver intention recognition and prediction algorithm, and designing a sensor-switching strategy to improve the local consensus-based bundle algorithm (LCBBA). Firstly, a recognizable maneuver intention model that can describe the maneuver types and directions of the HGVs in both the longitudinal and lateral directions was designed. Secondly, a maneuver intention recognition and prediction algorithm based on parallel, stacked long short-term memory neural networks (PSLSTM) was developed to obtain maneuver directions of the HGV. On the basis of that, a satellite constellation tracking decision method (referred to as SS-LCBBA in the following) considering the HGVs’ maneuver intentions was designed. Finally, the maneuver intention prediction capability of the PSLSTM network and two currently popular network structures: the multilayer LSTM (M-LSTM) and the dual-channel and bidirectional neural network (DCBNN) were tested for comparison. The simulation results show that the PSLSTM can recognize and predict the maneuver directions of HGVs with high accuracy. In the simulation of a satellite constellation tracking HGVs, the SS-LCBBA improved the cumulative tracking score compared to the LCBBA, the blackboard algorithm (BM), and the variable-center contract network algorithm (ICNP). Thus, it is concluded that SS-LCBBA has better adaptability to environments with uncertain intentions in solving multi-satellite collaborative scheduling problems.

2.5D Laser-Cutting-Based Customized Fabrication of Long-Term Wearable Textile sEMG Sensor: From Design to Intention Recognition

The surface electromyography(sEMG) sensor is widely used as a human-machine interface in wearable systems. Although numerous studies have applied compact sEMG systems to wearable devices, these are inconvenient and not suitable for long-term use. Herein, we introduce a 2.5D laser cutting method to accelerate customized sensor fabrication from design to production. The customized textile-based sensor provides high wearing comfort and improves the sensor signal quality through stable contact. We implemented a foam-filled electrode to ensure solid skin-electrode contact even during perspiration and varying pressure conditions, and evaluated its performance experimentally. The sensor-integrated garments one for the leg and one for the arm were fabricated with the proposed design method for further evaluation and application. Consistent sensor performance was demonstrated during squats and running (i.e., perspiration) while wearing the leg sensor. The sensor sleeve for the arm was integrated with intention recognition algorithms for hand gesture recognition. A Convolutional Neural Network (CNN) architecture was employed to classify 28 hand gestures, including finger and wrist motions. The average classification accuracy of five subjects achieved 93.21%, and further increased to 94.34% after perspiration.

Development of vehicle maneuvering system for autonomous driving

Recently, autonomous driving technology has received significant interest, and consequently, various autonomous driving algorithms have been developed by researchers. An experimental (conventional) vehicle verifies the strategy for autonomous driving by testing the performance and feasibility of the corresponding algorithms. Implementation of the autonomous driving strategy in the conventional vehicle system requires modification of the electronic control unit (ECU); however, this is limited by security issues and the high cost of development.In this paper, a maneuvering system that can control the steering angle, braking force, and accelerating signal so that autonomous driving does not operate via the ECU is proposed. Because the steering angle is an important factor in terms of the vehicle motion behavior, a robust angle controller based on the disturbance observer is adopted. Moreover, to ensure safety and user convenience, a user intention recognition algorithm is proposed. Based on this algorithm, the maneuvering system can be disabled during control without using any perception sensor. Finally, several experiments are conducted to evaluate the functions of the maneuvering system. A scenario-based driving test is also performed to verify the feasibility of the maneuvering system.

Task Offloading and Scheduling Strategy for Intelligent Prosthesis in Mobile Edge Computing Environment

Traditional intent recognition algorithms of intelligent prosthesis often use deep learning technology. However, deep learning’s high accuracy comes at the expense of high computational and energy consumption requirements. Mobile edge computing is a viable solution to meet the high computation and real-time execution requirements of deep learning algorithm on mobile device. In this paper, we consider the computation offloading problem of multiple heterogeneous edge servers in intelligent prosthesis scenario. Firstly, we present the problem definition and the detail design of MEC-based task offloading model for deep neural network. Then, considering the mobility of amputees, the mobility-aware energy consumption model and latency model are proposed. By deploying the deep learning-based motion intent recognition algorithm on intelligent prosthesis in a real-world MEC environment, the effectiveness of the task offloading and scheduling strategy is demonstrated. The experimental results show that the proposed algorithms can always find the optimal task offloading and scheduling decision.

Research on Spoken Language Understanding Based on Deep Learning

Aiming at solving the problem that the recognition effect of rare slot values in spoken language is poor, which affects the accuracy of oral understanding task, a spoken language understanding method is designed based on deep learning. The local features of semantic text are extracted and classified to make the classification results match the dialogue task. An intention recognition algorithm is designed for the classification results. Each datum has a corresponding intention label to complete the task of semantic slot filling. The attention mechanism is applied to the recognition of rare slot value information, the weight of hidden state and corresponding slot characteristics are obtained, and the updated slot value is used to represent the tracking state. An auxiliary gate unit is constructed between the upper and lower slots of historical dialogue, and the word vector is trained based on deep learning to complete the task of spoken language understanding. The simulation results show that the proposed method can realize multiple rounds of man-machine spoken language. Compared with the spoken language understanding methods based on cyclic network, context information, and label decomposition, it has higher accuracy and F1 value and has higher practical application value.

Benchmark of public intent recognition services

The intent recognition is an essential algorithm of any conversational AI application. It is responsible for the classification of an input message into meaningful classes. In many bot development platforms, we can configure the NLU pipeline. Several intent recognition services are currently available as an API, or we choose from many open-source alternatives. However, there is no comparison of intent recognition services and open-source algorithms. Many factors make the selection of the right approach to the intent recognition challenging in practice. In this paper, we suggest criteria to choose the best intent recognition algorithm for an application. We present a dataset for evaluation. Finally, we compare selected public NLU services with selected open-source algorithms for intent recognition.

Online Risk-Bounded Motion Planning for Autonomous Vehicles in Dynamic Environments

A crucial challenge to efficient and robust motion planning for autonomous vehicles is understanding the intentions of the surrounding agents. Ignoring the intentions of the other agents in dynamic environments can lead to risky or overconservative plans. In this work, we model the motion planning problem as a partially observable Markov decision process (POMDP) and propose an online system that combines an intent recognition algorithm and a POMDP solver to generate risk-bounded plans for the ego vehicle navigating with a number of dynamic agent vehicles. The intent recognition algorithm predicts the probabilistic hybrid motion states of each agent vehicle over a finite horizon using Bayesian filtering and a library of pre-learned maneuver motion models. We update the POMDP model with the intent recognition results in real time and solve it using a heuristic search algorithm which produces policies with upper-bound guarantees on the probability of near colliding with other dynamic agents. We demonstrate that our system is able to generate better motion plans in terms of efficiency and safety in a number of challenging environments including unprotected intersection left turns and lane changes as compared to the baseline methods.

Multimodal Sensor Motion Intention Recognition Based on Three-Dimensional Convolutional Neural Network Algorithm.

With the development of microelectronic technology and computer systems, the research of motion intention recognition based on multimodal sensors has attracted the attention of the academic community. Deep learning and other nonlinear neural network models have a wide range of applications in big data sets. We propose a motion intention recognition algorithm based on multimodal long-term and short-term spatiotemporal feature fusion. We divide the target data into multiple segments and use a three-dimensional convolutional neural network to extract the short-term spatiotemporal features. The three types of features of the same segment are fused together and input into the LSTM network for time-series modeling to further fuse the features to obtain multimodal long-term spatiotemporal features with higher discrimination. According to the lower limb movement pattern recognition model, the minimum number of muscles and EMG signal characteristics required to accurately recognize the movement state of the lower limbs are determined. This minimizes the redundant calculation cost of the model and ensures the real-time output of the system results.

Denoising Algorithm for Event-Related Desynchronization-Based Motor Intention Recognition in Robot-assisted Stroke Rehabilitation Training with Brain-Machine Interaction

BackgroundRehabilitation robots integrated with brain-machine interaction (BMI) can facilitate stroke patients’ recovery by closing the loop between motor intention and actual movement. The main challenge is to identify the patient’s motor intention based on large training datasets with noise contamination in the Electroencephalogram (EEG) signal. New methodTo address this problem, this paper proposed a self-adaptively denoised Event-Related Desynchronization (ERD)-based motor intention recognition algorithm (DeERD) in order to enable BMI training with a small sample of calibration data. This study recruited 8 stroke patients. Each patient was required to execute paralyzed upper-limb motor attempt for 20 trials and remain in resting state for 20 trials randomly. ERD-based motor intention recognition algorithm, Common spatial filter algorithm (CSP) and Directed Transfer Function analysis (DTF) were used to extract features for classification respectively and compared with the proposed DeERD analysis. ResultsDeERD can filter the noise and extract the average lines as the principal trends. With denoising processing, Accuracy (ACC) was up to 70% for all 8 patients and they could be included in this BMI system effectively. Comparison with existing methodsThe proposed DeERD model generated statistically significant increase in True Positive Rate (TPR) and in ACC than the DTF model. TPR and ACC standard deviation of DeERD was smaller than that of CSP. ConclusionsThe proposed DeERD model can eliminate the principal noise and extract the principal trend of the time-frequency analysis. It provides a practical method to recruit more stroke patients into BMI training with fewer calibration trainings.

Driver’s intention recognition algorithm based on recessive Markoff model

Driver’s intention recognition algorithm based on recessive Markoff model

Research on Human-Computer Interaction Intention Recognition Based on EEG and Eye Movement

In this work, we present a novel method to intention recognition, based on electroencephalogram (EEG) and eye movement in human-computer interaction(HCI). The fusion of EEG and eye movement will allow the utmost of the advantages of the two physiological signals. Signals of EEG and eye movement were collected for feature extraction, recognition network of machine learning pattern was input for intent recognition, final recognition result was attained by decision-level fusion. We compare the results of the Intention Recognition Algorithms to those of an experiment involving the intention recognition of the operator in a simulated flight mission. In most every case, results show that the intention recognition algorithms performed better than solely rely on single signal.

Man-machine interaction-based motion control of a robotic walker.

The aging population brings the problem of healthcare and dyskinesia. The lack of mobility extremely affects stroke patient's activities of daily living (ADL) and decreases their quality of life. To assist these mobility-limited people, a robotic walker is designed to facilitate gait rehabilitation training. The aim of this paper is to present the implementation of a novel motion control method to assist disabled people based on their motion intention. The kinematic framework of the robotic walker is outlined. We propose an intention recognition algorithm based on the interactive force signal. A novel motion control method combined with T-S fuzzy controller and PD controller is proposed. The motion controller can recognize the intention of the user through the interactive force, which allows the user to move or turn around as usual, instead of using their hands to control the walker. Preliminary experiments with healthy individuals and simulated patients are carried out to verify the effectiveness of the algorithm. The results show that the proposed motion control approach can recognize the user's intention, is easy to control and has a higher precision than the traditional proportional-integral-derivative controller. The results show that users could achieve the task with acceptable error, which indicates the potential of the proposed control method for gait training.

Virtual Reality Robot-Assisted Welding Based on Human Intention Recognition

We propose an innovative approach to enhance welding operations by using a cyber-physical system (CPS) with layered architecture and enabling a robot to be effectively operated by its commanding human. This article focuses on the recognition of the commanding human’s intention that should be executed by the robot. To this end, a virtual reality (VR) system based on the HTC Vive is used to create a remote virtual welding environment. Human hand movement speed data is collected and used to train a hidden Markov model (HMM) using the Baum-Welch algorithm. The Bayesian information criterion (BIC) is applied to determine the number of hidden states. The state occupancy probability distribution (the probability of each state at a given time) is estimated based on the human hand movement speed sequence using the forward algorithm. The human intention, defined as the intended movement in this article, is then estimated as the statistical expectation of the observable variables. Using the proposed human intention estimation algorithm, the intended movement recognized from the raw movement data is smoother, which is preferable in welding tasks. A 6-DoF industrial robot, UR-5 with a custom gas tungsten arc welding (GTAW) torch installed, works as the final performer of the welding jobs. The robot receives the intended movement data from the HMM and uses this to assist the human welding operators. Welding experiments have been conducted both with and without the proposed human intention recognition (IR) algorithm. The results show that the robot can help the operators complete welding tasks with better performance using the proposed IR system, supporting the effectiveness of the proposed VR robot-assisted welding system. Note to Practitioners —Welding is not only labor-intensive but also requires real-time adaption to the process, which is challenging for robots/machines but relatively straightforward for humans. Using a human commanded robot to perform welding can liberate humans from laborious operations and hazardous environments. To this end, a virtual reality (VR) system is used to create a virtual welding environment for a human to view the process remotely and for a human to pass his/her resultant adaptation to the robot through hand movements. However, hand movements do not always fully represent the intended adaptation of the commanding human, and the recognition of such human intention is fundamental in such a proposed method. This article established the mathematical framework for the recognition of the human intention and, thus, the foundation for effective assistance of robots to humans.

Braking intention recognition algorithm based on electronic braking system in commercial vehicles

The aim of this research is to investigate the braking intention identification method adaptive to the electronic braking system (EBS) in commercial vehicles. Based on the neural network, a braking intention identification model is established which takes both emergency braking and general braking into account. Then, considering the complex transportation environment, a multi-condition identification model with respect to four typical braking conditions is developed using the fuzzy logic. The experimental results of the two models demonstrate that the proposed strategy can make good use of driver braking intention. The proposed method provides theoretical guidelines on driver behaviour adaptation on the longitudinal active safety system, which promotes vehicle safety and braking performance.

Braking intention recognition algorithm based on electronic braking system in commercial vehicles

Braking intention recognition algorithm based on electronic braking system in commercial vehicles