Human-machine Interactive Control Research Articles

Research on Human-Machine Interaction Control Technology for Lower Limb Exoskeleton Rehabilitation Robots

In response to the health assurance challenges faced by the elderly population in Chinas aging society and to mitigate the shortage of medical resources while improving the rehabilitation rate of patients with central nervous system injuries, research on lower limb exoskeleton rehabilitation robots has been widely conducted. Since the 1960s, countries such as Switzerland, the United States, the Netherlands, Germany, Japan, and China have successively developed relatively mature standing and recumbent lower limb exoskeleton systems. This paper builds the kinematic and dynamic models of a lower limb exoskeleton rehabilitation robot based on a two-link mechanism, laying the theoretical foundation and implementation approach for processing position, velocity, and torque sensor signals in human-machine interaction control. Based on domestic and international progress as well as kinematic and dynamic analysis, four types of human-machine interaction control technologiestrajectory tracking control, force-position hybrid control, impedance control, and bioelectrical signal controlare thoroughly analyzed, providing a technical framework for engineering implementation. In the future, lower limb exoskeleton rehabilitation robots will integrate lightweight materials and structural design, intelligent adaptive algorithms, and environmental perception technologies, utilizing low-cost open-source platforms to optimize human-machine interaction control technology for more efficient, safe, and personalized rehabilitation training

High-performance gelatin-based hydrogel flexible sensor for respiratory monitoring and human–machine interaction

Natural hydrogels like gelatin are ideal for fabricating sensors that monitor human body signals due to their excellent biocompatibility. However, their typically large molecular weight restricts molecular mobility and repositioning under stress, limiting their stretchability performance. In this study, a hydrogel sensor PAM-Gel/β-GP/LiCl (named: PGBL) combining gelatin with ion salts, is proposed. Through the synergistic effect of sodium β-glycerophosphate (β-GP) and LiCl, the PAM-gelatin-based hydrogel achieves an extraordinary elongation strain exceeding 11000 %, enabling ultra-stretchability. Additionally, PGBL exhibits excellent electrical conductivity (8.2 S/m), high sensitivity (GF approximately 4.1), and resilience to low temperatures (−24 °C). Moreover, PGBL demonstrates strong adhesion, making it suitable for skin attachment in human body sensing applications. Integrating PGBL hydrogel sensors with a robotic hand has led to the development of a human–machine interaction control system. Furthermore, combining real-time data transmission and visualization technologies has resulted in a real-time respiratory monitoring system, which can monitor sleep apnoea blockage. PGBL hydrogel sensors show promising applications in biomedical fields and biosensing, highlighting their potential in healthcare monitoring systems.

Ankle Moment Estimation Based on A Novel Distributed Plantar Pressure Sensing System.

Ankle moment plays an important role in human gait analysis, patients' rehabilitation process monitoring, and the human-machine interaction control of exoskeleton robots. However, current ankle moment estimation methods mainly rely on inverse dynamics (ID) based on optical motion capture system (OMC) and force plate. These methods rely on fixed instruments in the laboratory, which are difficult to be applied to the control of exoskeleton robots. To solve this problem, this paper developed a new distributed plantar pressure system and proposed an ankle plantar flexion moment estimation method using the plantar pressure system. We integrated eight pressure sensors in each insole to collect the pressure data of the key area of the foot and then used the plantar pressure data to train four neural networks to obtain the ankle moment. The performance of the models was evaluated using normalized root mean square error (NRMSE) and cross-correlation coefficient (ρ). During experiments, eight subjects were recruited for the overground walking tests, and OMC and force plate were used as the gold standard. The results indicate that the Genetic algorithm - Gated recurrent unit estimation algorithm (GA-GRU) was the best estimation model which achieved the highest accuracy in generalized ankle moment estimation (NRMSE = 7.23%, ρ = 0.85) compared with the other models. The designed novel distributed plantar pressure system and the proposed method could serve as a joint moment estimation approach in wearable robot control and human motion state monitoring.

Bio-inspired e-skin with integrated antifouling and comfortable wearing for self-powered motion monitoring and ultra-long-range human-machine interaction

Electronic skin (e-skin) inspired by the sensory function of the skin demonstrates broad application prospects in health, medicine, and human–machine interaction. Herein, we developed a self-powered all-fiber bio-inspired e-skin (AFBI E-skin) that integrated functions of antifouling, antibacterial, biocompatibility and breathability. AFBI E-skin was composed of three layers of electrospun nanofibrous films. The superhydrophobic outer layer Poly(vinylidene fluoride)-silica nanofibrous films (PVDF-SiO2 NFs) possessed antifouling properties against common liquids in daily life and resisted bacterial adhesion. The polyaniline nanofibrous films (PANI NFs) were used as the electrode layer, and it had strong “static” antibacterial capability. Meanwhile, the inner layer Polylactic acid nanofibrous films (PLA NFs) served as a biocompatible substrate. Based on the triboelectric nanogenerator principle, AFBI E-skin not only enabled self-powered sensing but also utilized the generated electrical stimulation for “dynamic” antibacterial. The “dynamic-static” synergistic antibacterial strategy greatly enhanced the antibacterial effect. AFBI E-skin could be used for self-powered motion monitoring to obtain a stable signal output even when water was splashed on its surface. Finally, based on AFBI E-skin, we constructed an ultra-long-range human-machine interaction control system, enabling synchronized hand gestures between human hand and robotic hand in any internet-covered area worldwide theoretically. AFBI E-skin exhibited vast application potential in fields like smart wearable electronics and intelligent robotics.

Printed-scalable microstructure BaTiO3/ecoflex nanocomposite for high-performance triboelectric nanogenerators and self-powered human-machine interaction

Triboelectric sensing technology reinforced with deep learning algorithm has shown significant potentials in the fields of sophisticated wearable devices and Internet of Things (IoT) applications. Herein, we present a large-scale and printable nanocomposite film for high-performance microstructured BaTiO3/Ecoflex triboelectric nanogenerator (MBE-TENG) and demonstrate its applications in deep-learning assisted self-powered sensing and human-machine interaction. The MBE-TENG device achieves a short-circuit current of 1.46 μA, an open-circuit voltage of 144 V, and a transfer charge density of 6.62 nC/cm². Additionally, the MBE-TENG device can charge a 10 μF capacitor from 0 to 2 V within 200 seconds, illuminate over 70 commercial LEDs, and power a digital calculator. Based on the MBE-TENG, a sensory array keyboard is demonstrated to wirelessly control the movement of a smart car via Bluetooth communication. Furthermore, a smart tactile sensing glove is developed by combining the MEB-TENG, residual neural network and convolutional block attention module, realizing the facile recognition on six different objects with a high accuracy of 96.33 %. This work presents a scalable fabrication for microstructured triboelectric layer and highlights the versatility and efficiency of TENG sensing technology when combined with advanced neural networks and embedded systems, paving the way for innovative development in human-machine interaction, personalized healthcare, and intelligent control interfaces.

An Image-Based Interactive Training Method of an Upper Limb Rehabilitation Robot

Aimed at the problem of human–machine interaction between patients and robots in the process of using rehabilitation robots for rehabilitation training, this paper proposes a human–machine interactive control method based on an independently developed upper limb rehabilitation robot. In this method, the camera is used as a sensor, the human skeleton model is used to analyse the moving image, and the key points of the human body are extracted. Then, the three-dimensional coordinates of the key points of the human arm are extracted by depth estimation and spatial geometry, and then the real-time motion data are obtained, and the control instructions of the robot are generated from it to realise the real-time interactive control of the robot. This method can not only improve the adaptability of the system to individual patient differences, but also improve the robustness of the system, which is less affected by environmental changes. The experimental results show that this method can realise real-time control of the rehabilitation robot, and that the robot assists the patient to complete the action with high accuracy. The results show that this control method is effective and can be applied to the fields of robot control and robot-assisted rehabilitation training.

SEMG data driven-based anti-disturbance control enables adaptive interaction of lower limb rehabilitation exoskeleton

The efficient human–machine interaction control is the core of human–machine integration, which can improve the tracking accuracy of the lower limb rehabilitation exoskeleton system and the compliance of human–machine cooperative movement. However, the non-ideal factors such as mechanical friction, model uncertainties, iteration errors and external interference in the training process undoubtedly bring potential dangers to the safety of rehabilitation training. Consequently, to escape unnecessary damage caused by external disturbances (especially the unbounded disturbances) during rehabilitation training, a noise-suppressing zeroing neural network human–machine interaction controller is developed in this work, which is based on the constructed human–machine coupling dynamic model and the active motion intention (active torque) of the subject identified by exploiting the deep convolutional neural network. The simulation experiments and statistical analyses verify that the present noise-suppressing zeroing neural network controller can be applied to monitor the lower limb rehabilitation exoskeleton for assisting the subjects in various task with the external disturbances,and the average root mean square error of the hip, knee and ankle joints’ angles are 0.0015rad, 0.0051rad and 0.0056rad, respectively. Furthermore, a novel model predictive control is developed and analyzed based on noise-suppressing zeroing neural network controller, which can effectively constrain the angle and angular velocity of the lower limb rehabilitation exoskeleton. The root mean square errors of hip, knee and ankle joint angle are 0.0032rad, 0.0078rad, and 0.0085rad, respectively. Finally, the platform experiment verifies that the proposed controllers can be utilized to control the lower limb rehabilitation exoskeleton to assist the subjects in rehabilitation training.

Human–machine interaction controller of upper limb based on iterative learning method with zeroing neural algorithm and disturbance observer

In this paper, an iterative learning controller with zeroing neural algorithm and disturbance observer is proposed to solve the conflicting of human–machine interaction and disturbances in system. This paper presents a theoretical framework, which is able to process rigorous stability analysis of human–machine interaction control. The iterative learning controller is suitable for repetitive upper limb rehabilitation training. Combining tracking error dependent weight vector with zeroing neural algorithm is aimed to reduce the conflicts in various operations of motions between upper limb and machine. The disturbance observer is used to deal with eliminating the disturbance. In addition, simulations of the controller can effectively and safety assist upper limb movement in different training stages.

Research on optimization of human-machine interaction control strategy for exoskeleton based on impedance control

Research on optimization of human-machine interaction control strategy for exoskeleton based on impedance control

Fuzzy model reference learning control in surgical robots

This paper proposes to apply the fuzzy model reference learning control method based on the guide control to the collaborative human-machine dragging process of the surgical robot. Variable damping coefficient adjustment parameter rules for fuzzy conductance controllers are trained by an offline learning mechanism in order to achieve the best human-machine interaction control effect. The surgical risk caused by the doctor's misoperation is lowered. Through the simulation experiment results, the speed curve tracking error is reduced by 70%. Trajectory curve tracking error is reduced by 57%. The both results prove that the strategy can reduce the error of the surgical robot and ensure the safety of robotic surgical operation.

A noise-suppressing neural network approach for upper limb human-machine interactive control based on sEMG signals.

The use of upper limb rehabilitation robots to assist the affected limbs for active rehabilitation training is an inevitable trend in the field of rehabilitation medicine. In particular, the active motion intention-based control of the upper limb rehabilitation robots to assist subjects in rehabilitation training is a hot research topic in human-computer interaction control. Therefore, improving the accuracy of active motion intention recognition is the premise of the human-machine interaction controller design. Furthermore, there are external disturbances (bounded/unbounded disturbances) during rehabilitation training, which seriously threaten the safety of subjects. Thereby, eliminating external disturbances (especially unbounded disturbances) is the difficulty and key to the human-machine interaction control of the upper limb rehabilitation robots. In response to these problems, based on the surface electromyogram signal of the human upper limb, this paper proposes a fuzzy neural network active motion intention recognition method to explore the internal connection between the surface electromyogram signal of the human upper limb and active motion intention, and improve the real-time and accuracy of recognition. Based on this, two types of human-machine interaction controllers, which can be called as zeroing neural network controller and noise-suppressing zeroing neural network controller are designed to establish a safe and comfortable training environment to avoid secondary damage to the affected limb. Numerical experiments verify the feasibility and effectiveness of the proposed theories and methods.

Porous-Structure-Promoted Tribo-Induced High-Performance Self-Powered Tactile Sensor toward Remote Human-Machine Interaction.

Self-powered tactile sensor with versatile functions plays a significant role in the development of an intelligent human-machine interaction (HMI) system. Herein, a hybrid self-powered porous-structured tactile sensor (SPTS) is proposed by monolithically integrating a porous triboelectrification-induced electroluminescence (TIEL) component and a single-electrode triboelectric nanogenerator with the high charge generation in the bulk volume. At a low pressure of 10kPa, TIEL intensity can be significantly improved by three times, which is superior to that in previous reports, with enhanced triboelectricity. Based on the enhancement brought by the porous structure and optimized parameters, the SPTS achieves significant sensing performance in both optical and electrical modes. To demonstrate the potential of practical applications, a programmable optical and electrical dual-mode HMI system is established based on SPTS to remotely control an intelligent vehicle and operate a computer game through identifying finger touch trajectories. This work not only contributes a new economical-effective methodology toward a high-performance tribo-induced self-powered tactile sensor but also facilitates the remote control of HMI with dual-mode functionality, which has broad potential applications in the fields of intelligent robots, augmented reality, flexible wearable electronics, and smart home.

Improving the Robustness of Human-Machine Interactive Control for Myoelectric Prosthetic Hand During Arm Position Changing.

Robust classification of natural hand grasp type based on electromyography (EMG) still has some shortcomings in the practical prosthetic hand control, owing to the influence of dynamic arm position changing during hand actions. This study provided a framework for robust hand grasp type classification during dynamic arm position changes, improving both the “hardware” and “algorithm” components. In the hardware aspect, co-located synchronous EMG and force myography (FMG) signals are adopted as the multi-modal strategy. In the algorithm aspect, a sequential decision algorithm is proposed by combining the RNN-based deep learning model with a knowledge-based post-processing model. Experimental results showed that the classification accuracy of multi-modal EMG-FMG signals was increased by more than 10% compared with the EMG-only signal. Moreover, the classification accuracy of the proposed sequential decision algorithm improved the accuracy by more than 4% compared with other baseline models when using both EMG and FMG signals.

Decoding finger movement patterns from microscopic neural drive information based on deep learning

Recent development of surface electromyogram (sEMG) decomposition technique provides a good basis of decoding movements from individual motor unit (MU) activities that directly representing microscopic neural drives. How to interpret the function and contribution of each decomposed MU to macroscopic movements remains unclear. The objective of this study is to decode finger movement patterns by establishing a relationship between individual MU activities and movements. In this study, high-density sEMG (HD-sEMG) data were recorded by a 16 × 8 electrode array from finger extensor muscles of 10 subjects performing 10 finger movement patterns. The progressive FastICA peel-off algorithm was first applied to decompose the HD-sEMG data to obtain microscopic neural drives in terms of MU firing sequences and their corresponding action potential waveforms. Then, convolutional neural network was used for classification of the decomposed MUs by characterizing their spatial waveforms spanned over all channels of the array. On this basis, a fuzzy weighted decision strategy was designed to give a final decision of movement pattern recognition, where function of an individual MU was measured in the form of contributing into all movement patterns with different weights to solve the issue of MUs shared among multiple patterns due to muscle co-activation. The proposed method yielded an average accuracy approximating to 100%, and it outperformed other common MU-based methods or conventional myoelectric classification methods using macroscopic sEMG features (p < 0.05). The proposed method has a wide application prospect in the field of human-machine interaction and precise motor control.

Musculoskeletal Joint Angle Estimation Based on Isokinetic Motor Coordination

Motion estimation plays an important role in motor coordination exploration and optimal control of human-machine interaction. Muscular strength training, such as isokinetic exercise, aims to improve one’s muscular strength and endurance with adjustable resistance at a constant speed. To explore motion estimation and muscle coordination, this paper proposes a sEMG based musculoskeletal model to estimate joint angle trajectory of elbow isokinetic exercise to build the relationship between surface-Electromyography (sEMG) and its related joint movement. A set of motion protocol is designed to evaluate the model, including isokinetic movements under 100% and 50% joint torque levels (TL), and isotonic exercise. sEMG sensors and isokinetic dynamometer are employed in which muscle activations are calculated from sEMG signals as the input of musculoskeletal model. The experiment of three motion protocols is conducted by seven healthy subjects. The synergy patterns indicate the muscle coordination consistency with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{R}^{2} {&gt;}$ </tex-math></inline-formula> 0.92. The estimated joint angle trajectory confirms that the musculoskeletal model predicts the isokinetic movement and isotonic movement accurately with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{R}^{2} {&gt;}$ </tex-math></inline-formula> 0.90 and the normalized root mean squared deviation NRMSD < 0.18. The research outcome in different levels of muscle forces paves the way for tailoring isokinetic exercise in clinical therapy such as stroke rehabilitation.

The Role of Surface Electromyography in Data Fusion with Inertial Sensors to Enhance Locomotion Recognition and Prediction.

Locomotion recognition and prediction is essential for real-time human–machine interactive control. The integration of electromyography (EMG) with mechanical sensors could improve the performance of locomotion recognition. However, the potential of EMG in motion prediction is rarely discussed. This paper firstly investigated the effect of surface EMG on the prediction of locomotion while integrated with inertial data. We collected EMG signals of lower limb muscle groups and linear acceleration data of lower limb segments from ten healthy participants in seven locomotion activities. Classification models were built based on four machine learning methods—support vector machine (SVM), k-nearest neighbor (KNN), artificial neural network (ANN), and linear discriminant analysis (LDA)—where a major vote strategy and a content constraint rule were utilized for improving the online performance of the classification decision. We compared four classifiers and further investigated the effect of data fusion on the online locomotion classification. The results showed that the SVM model with a sliding window size of 80 ms achieved the best recognition performance. The fusion of EMG signals does not only improve the recognition accuracy of steady-state locomotion activity from 90% (using acceleration data only) to 98% (using data fusion) but also enables the prediction of the next steady locomotion (∼370 ms). The study demonstrates that the employment of EMG in locomotion recognition could enhance online prediction performance.

Human–machine interaction control for stochastic cell manipulation systems

While many physical robot systems are able to work cooperatively with humans to perform numerous tasks, existing optical cell manipulation systems can only perform either manual or automated tasks at a time. As optical cell manipulation is essentially different from robotic manipulation in the physical world due to its stochastic nature, existing human–robot interaction control techniques cannot thus be directly reproduced into optical micro-manipulation of cells. This paper proposes a human–machine interaction control approach for stochastic optical cell manipulation systems. A theoretical foundation is then developed, and thus enabling human to interpose or interact, when applicable or needed, with an automated cell manipulation system without interrupting the manipulation process. Therefore, it offers an opportunity for human to get involved with an automated cell manipulation system so as to deal with unexpected events or complex tasks, rather than to stop or restart the whole manipulation process. Both the capacity of an automated control system – which provides precise and productive cell manipulation, and human ability – in terms of decision making, if needed, are then consolidated into a single optical cell manipulation system. Rigorous mathematical formulation is developed with the consideration of the stochastic perturbations, and the stability of the human–machine interaction control system is analyzed from a stochastic perspective. Experiments are then performed to illustrate the effectiveness of the proposed approach.

A New Projected Active Set Conjugate Gradient Approach for Taylor-Type Model Predictive Control: Application to Lower Limb Rehabilitation Robots With Passive and Active Rehabilitation.

In this paper, a three-order Taylor-type numerical differentiation formula is firstly utilized to linearize and discretize constrained conditions of model predictive control (MPC), which can be generalized from lower limb rehabilitation robots. Meanwhile, a new numerical approach that projected an active set conjugate gradient approach is proposed, analyzed, and investigated to solve MPC. This numerical approach not only incorporates both the active set and conjugate gradient approach but also utilizes a projective operator, which can guarantee that the equality constraints are always satisfied. Furthermore, rigorous proof of feasibility and global convergence also shows that the proposed approach can effectively solve MPC with equality and bound constraints. Finally, an echo state network (ESN) is established in simulations to realize intention recognition for human–machine interactive control and active rehabilitation training of lower-limb rehabilitation robots; simulation results are also reported and analyzed to substantiate that ESN can accurately identify motion intention, and the projected active set conjugate gradient approach is feasible and effective for lower-limb rehabilitation robot of MPC with passive and active rehabilitation training. This approach also ensures computational when disturbed by uncertainties in system.

Design of AGV human-machine interactive control system

The outdoor intelligent transportation of a pharmaceutical factory requires the workshop to put forward task requirements according to the situation and provide multiple AGVs that can meet both man-machine control and networked scheduling. On the basis of existing AGV, an AGV control method based on human-computer interaction is proposed. First, design the overall architecture of the control system, use the touch screen to design and develop the AGV mobile client and task request client to form an industrial wireless network. The establishment of a network enables the AGV mobile client, task request client and vehicle management system to achieve cross-platform communication. Based on this, the client system sets and controls the AGV, which effectively solves the AGV’s man-machine control and networking scheduling problems. After functional testing and performance verification, the system works stably, with good flexibility and scalability.

Conductive cold-resistant and elastic hydrogel: A potential bionic skin for human-machine interaction control over artificial limbs

This work enables conductive cold-resistant and elastic hydrogel for a new application in electric steering engine control, demonstrating a great potential as bionic skin for goal-directed control over artificial limbs. The photocrosslinking of acrylamide provides an elastic matrix that is functionally improved by physical crosslinking of K2CO3, carrageenan and locust bean gum. The hydrogel is highly elastic with the stress of 0.04 MPa at 650% strain, and can recover immediately after being stretched to 500% or pressed to 10% of its original length. It can keep its softness and conductivity at a low temperature of −10 °C. When an intermittent bending force is applied, it synchronously outputs the resistance variation signals between 1.1 and 0.9 times the original value. We take advantages of all these physical features to create a soft controller for the convenient manipulation of electric steering engine. We translate the bending-induced resistance variation signals into digital data of between 0 and 1023, which are then encoded to direct the electric steering engine for various controlled motility. A simple bending of hydrogel can activate the electric steering engine in 0.18 s, showing a highly-synchronous control over the dynamic response of electric steering engine in many models.