Intelligent Human-Machine Systems Research Articles

Design of intelligent human-machine collaborative robot-assisted craniotomy system

Design of intelligent human-machine collaborative robot-assisted craniotomy system

Human-in-the-Loop Trajectory Optimization Based on sEMG Biofeedback for Lower-Limb Exoskeleton.

Lower-limb exoskeletons (LLEs) can provide rehabilitation training and walking assistance for individuals with lower-limb dysfunction or those in need of functionality enhancement. Adapting and personalizing the LLEs is crucial for them to form an intelligent human-machine system (HMS). However, numerous LLEs lack thorough consideration of individual differences in motion planning, leading to subpar human performance. Prioritizing human physiological response is a critical objective of trajectory optimization for the HMS. This paper proposes a human-in-the-loop (HITL) motion planning method that utilizes surface electromyography signals as biofeedback for the HITL optimization. The proposed method combines offline trajectory optimization with HITL trajectory selection. Based on the derived hybrid dynamical model of the HMS, the offline trajectory is optimized using a direct collocation method, while HITL trajectory selection is based on Thompson sampling. The direct collocation method optimizes various gait trajectories and constructs a gait library according to the energy optimality law, taking into consideration dynamics and walking constraints. Subsequently, an optimal gait trajectory is selected for the wearer using Thompson sampling. The selected gait trajectory is then implemented on the LLE under a hybrid zero dynamics control strategy. Through the HITL optimization and control experiments, the effectiveness and superiority of the proposed method are verified.

Surface electromyogram, kinematic, and kinetic dataset of lower limb walking for movement intent recognition

Surface electromyogram (sEMG) offers a rich set of motor information for decoding limb motion intention that serves as a control input to Intelligent human-machine synergy systems (IHMSS). Despite growing interest in IHMSS, the current publicly available datasets are limited and can hardly meet the growing demands of researchers. This study presents a novel lower limb motion dataset (designated as SIAT-LLMD), comprising sEMG, kinematic, and kinetic data with corresponding labels acquired from 40 healthy humans during 16 movements. The kinematic and kinetic data were collected using a motion capture system and six-dimensional force platforms and processed using OpenSim software. The sEMG data were recorded using nine wireless sensors placed on the subjects’ thigh and calf muscles on the left limb. Besides, SIAT-LLMD provides labels to classify the different movements and different gait phases. Analysis of the dataset verified the synchronization and reproducibility, and codes for effective data processing are provided. The proposed dataset can serve as a new resource for exploring novel algorithms and models for characterizing lower limb movements.

Adaptive Algorithm of the Wheelchair-Verticalizer User Interface System

Purpose of reseach. Development of a scheme and adaptive algorithm for the operation of an intelligent humanmachine system for the user interface of a wheelchair-verticalizer, taking into account the physiological characteristics of the operator, the wheeled platform and the external environment.Tasks. The study of the physiological characteristics of human-machine interaction in people with damage to the musculoskeletal system. Design development and mathematical description of the joystick as a component of a human-machine system. Development of adaptive algorithm and software for the wheelchair user interface system. Methods. Using the method of finite-state machine to describe the algorithm of motion modes switches. The use of polynomial functions in order to obtain smooth laws for changing the setpoints for the device drives. The use of nonlinear sensitivity coefficients of the joystick handle to provide adaptive modes of movement of the wheelchairverticalizer. Results. In the course of the study, the modes of movement of a wheelchair-verticalizer were developed and described. The design scheme and principle of operation of the joystick are presented and described. The modes of operation of the human-machine system was described mathematically and graphically. The adaptive algorithm of the user interface system of a wheelchair-verticalizer proposed in the paper interpret joystick inclinations into setpoints signals for wheelchair drive controllers. The use of non-linear sensitivity coefficients of the joystick handle to provide adaptive modes of operation of the wheelchair, taking into account the features of the hands movement of people with impaired functions of the musculoskeletal system, the movement of the wheelchair and the state of environment, is described.Conclusion. The adaptive algorithms of the human-machine system and the modes of movement of the upright wheelchair developed within the framework of the work make it possible to increase the safety and smoothness of movement through the use of non-linear sensitivity coefficients of the joystick handle and smooth laws of setting influences obtained on the basis of polynomial functions.

Predicting task performance for intelligent human-machine interactions.

Human-machine teams are deployed in a diverse range of task environments and paradigms that may have high failure costs (e.g., nuclear power plants). It is critical that the machine team member can interact with the human effectively without reducing task performance. These interactions may be used to manage the human's workload state intelligently, as the overall workload is related to task performance. Intelligent human-machine teaming systems rely on a facet of the human's state to determine how interaction occurs, but typically only consider the human's state at the current time step. Future task performance predictions may be leveraged to determine if adaptations need to occur in order to prevent future performance degradation. An individualized task performance prediction algorithm that relies on a multi-faceted human workload estimate is shown to predict a supervisor's task performance accurately. The analysis varies the prediction time frame (from 0 to 300 s) and compares results to a generalized algorithm.

Experimental research on efficacy of intelligent human-machine collaborative robot-assisted cranio-tomy system

Objective To explore the accuracy, efficiency and safety of robot-assisted craniotomy system named Cranibot. Methods The system called Cranibot was developed by Department of Neurosurgery, The First Medical Center of Chinese PLA General Hospital and Institute of Intelligent Robotics, Beijing Institute of Technology and then employed to carry out randomized controlled experiments on 3 kinds of subjects including 8 3D-printing PVC (polyvinyl chloride) skull models, 5 cadaveric pig heads and 5 living Bama miniature pigs. Both left and right sides were symmetrically selected in each subject to perform craniotomy, and on each side 4 holes were drilled to form a square bone flap. One part of the skull on one side was randomly categorized into the experimental group in which craniotomy was performed by system, and its symmetrical part of skull on the other side belonged to the control group in which craniotomy was performed by the same neurosurgeon team. We observed and measured the drilling position error, time spent on craniotomy and incidence of tissue injury in the two groups. Results In skull model experiments, the average position error and time spent on craniotomy of robot group were 1.87±0.66 mm and 6.64±1.15 min, and those of surgeon group were 3.14±0.73 mm and 8.06±1.10 min respectively. The differences were statistically significant (both P 0.05). Conclusion The robot-assisted craniotomy system named Cranibot could improve the accuracy and efficiency of craniotomy, while its safety requires further research with large sample size. Key words: Robotics; Neurosurgical procedures; Man-machine systems; Models, animal

Driving Anger States Detection Based on Incremental Association Markov Blanket and Least Square Support Vector Machine

Driving anger, known as “road rage”, has gradually become a serious traffic psychology issue. Although driving anger identification is solved in some studies, there is still a gap in driving anger grading which is helpful to take different intervening measures for different anger intensity, especially in real traffic environment. The main objectives of this study are: (1) explore a novel driving anger induction method based on various elicitation events, e.g., traffic congestion, vehicles weaving/cutting in line, jaywalking and red light waiting in real traffic environment; (2) apply incremental association Markov blanket (IAMB) algorithm to select typical features related to driving anger states; (3) employ least square support vector machine (LSSVM) to identify different driving anger states based on the selected features. Thirty private car drivers were enrolled to perform field experiments on a busy route selected in Wuhan, China, where drivers’ anger could be induced by the elicitation events within limited time. Meanwhile, three types of data sets including driver physiology, driving behaviors and vehicle motions, were collected by multiple sensors. The results indicate that 13 selected features including skin conductance, relative energy spectrum of β band of electroencephalogram, standard deviation (SD) of pedaling speed of gas pedal, SD of steering wheel angle rate, vehicle speed, SD of speed, SD of forward acceleration and SD of lateral acceleration have significant impact on driving anger states. The IAMB-LSSVM model achieves an accuracy with 82.20% which is 2.03%, 3.15%, 4.34%, 7.84% and 8.36% higher than IAMB using C4.5, NBC, SVM, KNN and BPNN, respectively. The results are beneficial to design driving anger detecting or intervening devices in intelligent human-machine systems.

An Optimization Strategy for Weighted Extreme Learning Machine based on PSO

Machine learning is a subfield of artificial intelligence concerned with techniques that allow computers to improve their outputs based on previous experiences. Among numerous machine learning algorithms, Weighted Extreme Learning Machine (WELM) is one of the famous cases recently. It not only has Extreme Learning Machine (ELM)’s extremely fast training speed and better generalization performance than traditional Neuron Network (NN), but also has the merit in handling imbalance data by assigning more weight to minority class and less weight to majority class. But it still has the limitation of its weight generated according to class distribution of training data, thereby, creating dependency on input data [R. Sharma and A. S. Bist, Genetic algorithm based weighted extreme learning machine for binary imbalance learning, 2015 Int. Conf. Cognitive Computing and Information Processing (CCIP) (IEEE, 2015), pp. 1–6; N. Koutsouleris, Classification/machine learning approaches, Annu. Rev. Clin. Psychol. 13(1) (2016); G. Dudek, Extreme learning machine for function approximation–interval problem of input weights and biases, 2015 IEEE 2nd Int. Conf. Cybernetics (CYBCONF) (IEEE, 2015), pp. 62–67; N. Zhang, Y. Qu and A. Deng, Evolutionary extreme learning machine based weighted nearest-neighbor equality classification, 2015 7th Int. Conf. Intelligent Human-Machine Systems and Cybernetics (IHMSC), Vol. 2 (IEEE, 2015), pp. 274–279]. This leads to the lack of finding optimal weight at which good generalization performance could be achieved [R. Sharma and A. S. Bist, Genetic algorithm based weighted extreme learning machine for binary imbalance learning, 2015 Int. Conf. Cognitive Computing and Information Processing (CCIP) (IEEE, 2015), pp. 1–6; N. Koutsouleris, Classification/machine learning approaches, Annu. Rev. Clin. Psychol. 13(1) (2016); G. Dudek, Extreme learning machine for function approximation–interval problem of input weights and biases, 2015 IEEE 2nd Int. Conf. Cybernetics (CYBCONF) (IEEE, 2015), pp. 62–67; N. Zhang, Y. Qu and A. Deng, Evolutionary extreme learning machine based weighted nearest-neighbor equality classification, 2015 7th Int. Conf. Intelligent Human-Machine Systems and Cybernetics (IHMSC), Vol. 2 (IEEE, 2015), pp. 274–279]. To solve it, a hybrid algorithm which composed by WELM algorithm and Particle Swarm Optimization (PSO) is proposed. Firstly, it distributes the weight according to the number of different samples, determines weighted method; Then, it combines the ELM model and the weighted method to establish WELM model; finally it utilizes PSO to optimize WELM’s three parameters (input weight, bias, the weight of imbalanced training data). Experiment data from both prediction and recognition show that it has better performance than classical WELM algorithms.

Optimal Geometrical Set for Automated Marker Placement to Virtualized Real-Time Facial Emotions.

In recent years, real-time face recognition has been a major topic of interest in developing intelligent human-machine interaction systems. Over the past several decades, researchers have proposed different algorithms for facial expression recognition, but there has been little focus on detection in real-time scenarios. The present work proposes a new algorithmic method of automated marker placement used to classify six facial expressions: happiness, sadness, anger, fear, disgust, and surprise. Emotional facial expressions were captured using a webcam, while the proposed algorithm placed a set of eight virtual markers on each subject’s face. Facial feature extraction methods, including marker distance (distance between each marker to the center of the face) and change in marker distance (change in distance between the original and new marker positions), were used to extract three statistical features (mean, variance, and root mean square) from the real-time video sequence. The initial position of each marker was subjected to the optical flow algorithm for marker tracking with each emotional facial expression. Finally, the extracted statistical features were mapped into corresponding emotional facial expressions using two simple non-linear classifiers, K-nearest neighbor and probabilistic neural network. The results indicate that the proposed automated marker placement algorithm effectively placed eight virtual markers on each subject’s face and gave a maximum mean emotion classification rate of 96.94% using the probabilistic neural network.

Optimizing the Flow Velocity of the Assembly Line: A Cognitive Neuroscience Perspective

Human resource is an indispensible element of industrial engineering. Based on Neuro-Industrial-Engineering, we combine the cognitive neuroscience and information technology with cybernetics, and advance a new intelligent human-machine system. Starting with neurophysiological signal (such as electroencephalogram) collection, the system is capable of real-time monitoring and online assessment of the workers’ physiological states. Based on the physiological information and the production information, the system could detect the bottleneck of the assembly line and realize automatic control of the assembly line’s flow velocity. It largely enhances the response speed of the system, and increase the productivity as well as product quality, which is much more effective than the Toyota system in which the assembly line is only stopped when a specific production accident has been detected during production.

Soft computing framework for intelligent human-machine system design, simulation and optimization

This paper proposes a novel soft-computing framework for human–machine system design and simulation based on the hybrid intelligent system techniques. The complex human–machine system is described by human and machine parameters within a comprehensive model. Based on this model, procedures and algorithms for human–machine system design, economical/ergonomic evaluation, and optimization are discussed in an integrated CAD and soft-computing framework. With a combination of individual neural and fuzzy techniques, the neuro-fuzzy hybrid soft-computing scheme implements a fuzzy if-then rules block for human–machine system design, evaluation and optimization by a trainable neural fuzzy network architecture. For training and test purposes, assembly tasks are simulated and carried out on a self-built multi-adjustable laboratory workstation with a flexible motion measurement and analysis system. The trained neural fuzzy network system is able to predict the operator's postures and joint angles of motion associated with a range of workstation configurations. It can also be used for design/layout and adjustment of human assembly workstations. The developed system provides a unified, intelligent computational framework for human–machine system design and simulation. Case studies for workstation system design and simulation are provided to illustrate and validate the developed system.

An intelligent human-machine system based on an ecological interface design concept

It seems both necessary and promising to develop an intelligent human-machine system, considering the objective of the human-machine system and the recent advance in cognitive engineering and artificial intelligence together with the ever-increasing importance of human factor issues in nuclear power plant operation and maintenance. It should support human operators in their knowledge-based behaviour and allow them to cope with unanticipated abnormal events, including recovery from erroneous human actions. A top-down design approach has been adopted based on cognitive work analysis, and (1) an ecological interface, (2) a cognitive model-based advisor and (3) a robust automatic sequence controller have been established. These functions have been integrated into an experimental control room. A validation test was carried out by the participation of experienced operators and engineers. The results showed the usefulness of this system in supporting the operator's supervisory plant control tasks.