Human-robot System Research Articles

Variable Admittance Control of High Compatibility Exoskeleton Based on Human–Robotic Interaction Force

The wearable exoskeleton system is a typical strongly coupled human–robotic system. Human–robotic is the environment for each other. The two support each other and compete with each other. Achieving high human–robotic compatibility is the most critical technology for wearable systems. Full structural compatibility can improve the intrinsic safety of the exoskeleton, and precise intention understanding and motion control can improve the comfort of the exoskeleton. This paper first designs a physiologically functional bionic lower limb exoskeleton based on the study of bone and joint functional anatomy and analyzes the drive mapping model of the dual closed-loop four-link knee joint. Secondly, an exoskeleton dual closed-loop controller composed of a position inner loop and a force outer loop is designed. The inner loop of the controller adopts the PID control algorithm, and the outer loop adopts the adaptive admittance control algorithm based on human–robot interaction force (HRI). The controller can adaptively adjust the admittance parameters according to the HRI to respond to dynamic changes in the mechanical and physical parameters of the human–robot system, thereby improving control compliance and the wearing comfort of the exoskeleton system. Finally, we built a joint simulation experiment platform based on SolidWorks/Simulink to conduct virtual prototype simulation experiments and recruited volunteers to wear rehabilitation exoskeletons to conduct related control experiments. Experimental results show that the designed physiologically functional bionic exoskeleton and adaptive admittance controller can significantly improve the accuracy of human–robotic joint motion tracking, effectively reducing human–machine interaction forces and improving the comfort and safety of the wearer. This paper proposes a dual-closed loop four-link knee joint exoskeleton and a variable admittance control method based on HRI, which provides a new method for the design and control of exoskeletons with high compatibility.

Classification of Co-Manipulation Modus with Human-Human Teams for Future Application to Human-Robot Systems

Despite the existence of robots that can physically lift heavy loads, robots that can collaborate with people to move heavy objects are not readily available. This article makes progress toward effective human-robot co-manipulation by studying 30 human-human dyads that collaboratively manipulated an object weighing \(27 \mathrm{kg}\) without being co-located (i.e., participants were at either end of the extended object). Participants maneuvered around different obstacles with the object while exhibiting one of four modi–the manner or objective with which a team moves an object together–at any given time. Using force and motion signals to classify modus or behavior was the primary objective of this work. Our results showed that two of the originally proposed modi were very similar, such that one could effectively be removed while still spanning the space of common behaviors during our co-manipulation tasks. The three modi used in classification were quickly , smoothly and avoiding obstacles . Using a deep convolutional neural network (CNN), we classified three modi with up to 89% accuracy from a validation set. The capability to detect or classify modus during co-manipulation has the potential to greatly improve human-robot performance by helping to define appropriate robot behavior or controller parameters depending on the objective or modus of the team.

Nonreciprocal interactions in crowd dynamics: Investigating the impact of moving threats on pedestrian speed preferences

Nonreciprocal interaction crowd systems, such as human–human, human–vehicle, and human–robot systems, often have serious impacts on pedestrian safety and social order. A more comprehensive understanding of these systems is needed to optimize system stability and efficiency. Despite the importance of these interactions, empirical research in this area remains limited. Thus, in our study we explore this underresearched area, focusing on scenarios where nonreciprocity plays a critical role, such as mass stabbings, which pose a substantial risk to public safety. We conducted the first experiments on this system and analysed high-accuracy data obtained from these experiments. Specifically, we conduct laboratory experiments on three scenarios: single exits, dual exits, and obscured chases. Then, we introduce an equation to describe the mechanism behind a direct threat zone, which significantly affects pedestrian behaviour. The extent of the direct threat zone is determined by the speed of the moving threat and the radius of danger occurrence. The equation can provide insights for different nonreciprocal interaction crowd systems, where the subjects of the nonreciprocal interactions can be a pedestrian, an autonomous robot or a vehicle. We further categorize potential threats into direct, adjacent, and rear-view zones, quantifying the level of threat for pedestrians. Our study revealed that a pedestrian’s desired velocity correlated positively with potential threat intensity, increasing until near the direct threat zone. An emerging steady state is observed when escape routes are blocked by moving threats. This deviation affects the density–velocity relationship, making it distinct from the general relationship. This deviation signifies unique pedestrian behaviour in the presence of moving threats. Additionally, the rate of change in the angle for pedestrian motion in various desired directions is synchronized. This indicates the emergence of collective intelligence in nonreciprocal interaction crowd systems. As a result, our study may constitute a pioneering step towards understanding nonreciprocal interactions in crowd systems through laboratory experiments. These findings may enhance pedestrian safety and inform not only government crowd management strategies but also individual self-protection measures.

Modeling and safety analysis for collaborative safety-critical systems using hierarchical colored Petri nets

ContextCollaborative systems enable multiple independent systems to work together towards a common goal. These systems can include both human-system and system-system interactions and can be found in a variety of settings, including smart manufacturing, smart transportation, and healthcare. Safety is an important consideration for collaborative systems because one system's failure can significantly impact the overall system performance and adversely affect other systems, humans or the environment. GoalFail-safe mechanisms for safety-critical systems are designed to bring the system to a safe state in case of a failure in the sensors or actuators. However, a collaborative safety-critical system must do better and be safe-operational, for e.g., a failure of one of the members in a platoon of vehicles in the middle of a highway is not acceptable. Thus, failures must be compensated, and compliance with safety constraints must be ensured even under faults or failures of constituent systems. MethodIn this paper, we model and analyze safety for collaborative safety-critical systems using hierarchical Coloured Petri nets (CPN). We used an automated Human Rescue Robot System (HRRS) as a case study, modeled it using hierarchical CPN, and injected some specified failures to check and confirm the safe behavior in case of unexpected scenarios. ResultsThe system behavior was observed after injecting three types of failures in constituent systems, and then safety mechanisms were applied to mitigate the effect of these failures. After applying safety mechanisms, the HRRS system's overall behavior was again observed both in terms of verification and validation, and the simulated results show that all the identified failures were mitigated and HRRS completed its mission. ConclusionIt was found that the approach based on formal methods (CPN modeling) can be used for the safety analysis, modeling, validation, and verification of collaborative safety-critical systems like HRRS. The hierarchical CPN provides a rigorous way of modeling to implement complex collaborative systems.

Local Observation Based Reactive Temporal Logic Planning of Human-Robot Systems

Local Observation Based Reactive Temporal Logic Planning of Human-Robot Systems

Performance of a Novel Muscle Synergy Approach for Continuous Motion Estimation on Untrained Motion (March 2024).

When applying continuous motion estimation (CME) model based on sEMG to human-robot system, it is inevitable to encounter scenarios in which the motions performed by the user are different from the motions in the training stage of the model. It has been demonstrated that the prediction accuracy of the currently effective approaches on untrained motions will be significantly reduced. Therefore, we proposed a novel CME method by introducing muscle synergy as feature to achieve better prediction accuracy on untrained motion tasks. Specifically, deep non-smooth NMF (Deep-nsNMF) was firstly introduced on synergy extraction to improve the efficiency of synergy decomposition. After obtaining the activation primitives from various training motions, we proposed a redundancy classification algorithm (RC) to identify shared and task-specific synergies, optimizing the original redundancy segmentation algorithm (RS). NARX neural network was set as the regression model for training. Finally, the model was tested on prediction tasks of eight untrained motions. The prediction accuracy of the proposed method was found to perform better than using time-domain feature as input of the network. Through Deep-nsNMF with RS, the highest accuracy reached 99.7%. Deep-nsNMF with RC performed similarly well and its stability was relatively higher among different motions and subjects. Limitation of the approach is that the problem of positive correlation between the prediction error and the absolute value of real angle remains to be further addressed. Generally, this research demonstrates the potential for CME models to perform well in complex scenarios, providing the feasibility of dedicating CME to real-world applications.

Task scheduling for heterogeneous agents pickup and delivery using recurrent open shop scheduling models

We study the transport-pick agents task scheduling (TPTS) problem in heterogeneous agents pickup and delivery (HAPD). Two functionally heterogeneous agent types, transport agents and pick agents, collaborate to execute multi-goal tasks subjecting to complex-schedule dependency. The objective is to plan a collective time-extended task schedule with the minimization of total completion time. To bridge the gap between robot task scheduling and general scheduling theory, a novel recurrent open shop scheduling (ROSS) problem variant with unique sequence structure is defined. New sequence and schedule models are extended to accommodate for it. Afterwards, the problem-specific append-beam-Christofides (ABC) constructive heuristic, greedy local search (GLS) and simulated annealing (SA) metaheuristic algorithms are designed accordingly. Theoretically, we rigorously analyze sequence and schedule structures, and algorithmic properties; empirically, we study the influence of different algorithm settings on a comprehensive dataset. Design guidelines and parameter settings of these algorithms are provided. The application conditions of the proposed methodology is discussed along with a baseline algorithm TEAMWISE. The proposed methodology could be utilized in various industrial heterogeneous multi-robot or collaborative human–robot systems.

Integrating Function Allocation and Operational Event Sequence Diagrams to Support Human-Robot Coordination: Case Study of a Robotic Date Thinning System

State-of-the-art robots show promise in supporting but not completely replacing human work in most precision agriculture applications. For many potential agricultural robot applications, there are no comparable systems nor readily available information on the human operator activities to guide the systems engineering process. Such is the situation for Medjool date thinning, a tedious and hazardous manual operation for which technological assistance has yet to be developed. Here we describe using cognitive system engineering methods to develop operational concepts and human-robot coordination requirements for a pioneer system, a Robotic Medjool Date Thinning System (RDTS). We leveraged the abstraction hierarchy to characterize the RDTS’s envisioned goals and functionality. We developed alternative functional allocations to explore the design space based on the availability of different enabling technologies. After downselecting to the function allocation, including the technologies expected to be developed, we created operational event sequence diagrams to visualize the operation flow and to identify requirements related to the human operator and the joint human-robot system. Applying these methods in the early design stages helped to refine the human-robot coordination requirements and to identify gaps in the operational concept—they show great potential to support the introduction of agricultural robots and bring them to fruition.

Human Robot Interaction: Applications, Challenges and Future Directions

Human-Robot Interaction (HRI) has grown over the years and is now used in a wide range of contexts including medicine, manufacturing, physical therapy, customer care, agriculture, and education. HRI is utilized in many workplace settings that entail co-manipulating tasks as well as picking and placing operations in production lines, welding processes, painting and parts assembly. It is also used in assistive robotics for people with physical and cognitive impairments. This study explores HRI as applied across different fields such as industry, health care, rehabilitating agricultures, service and education. The paper analyzes several types of HRI including geographically or temporally separated interactions, distant ones or those that are social in nature. Finally the research shows the advantages of HRI on productivity increase in quality enhancement competitiveness increase and improved working conditions. Lastly, this research delves into a number of challenges along with future prospects regarding issues such as safety precautions required while engaging in HRI activities; acceptable social behavior; distribution of work among others; there are other possible downsides to some of these emerging technologies on general well-being as well. To ensure safe and effective human robot systems more human factors investigations must be conducted in this area.

MIntNet: Rapid Motion Intention Forecasting of Coupled Human-Robot Systems With Simulation-to-Real Autoregressive Neural Networks

MIntNet: Rapid Motion Intention Forecasting of Coupled Human-Robot Systems With Simulation-to-Real Autoregressive Neural Networks

Development of a new set of Heuristics for the evaluation of Human-Robot Interaction in industrial settings: Heuristics Robots Experience (HEUROBOX).

Humans and robots will increasingly have to work together in the new industrial context. Therefore, it is necessary to improve the User Experience, Technology Acceptance, and overall wellbeing to achieve a smoother and more satisfying interaction while obtaining the maximum performance possible out of it. For this reason, it is essential to analyze these interactions to enhance User Experience. The heuristic evaluation is an easy-to-use, low-cost method that can be applied at different stages of a design process in an iterative manner. Despite these advantages, there is rarely a list of heuristics in the current literature that evaluates Human-Robot interactions both from a User Experience, Technology Acceptance, and Human-Centered approach. Such an approach should integrate key aspects like safety, trust, and perceived safety, ergonomics and workload, inclusivity, and multimodality, as well as robot characteristics and functionalities. Therefore, a new set of heuristics, namely, the HEUROBOX tool, is presented in this work in the form of the HEUROBOX tool to help practitioners and researchers in the assessment of human-robot systems in industrial environments. The HEUROBOX tool clusters design guidelines and methodologies as a logic list of heuristics for human-robot interaction and comprises four categories: Safety, Ergonomics, Functionality, and Interfaces. They include 84 heuristics in the basic evaluation, while the advanced evaluation lists a total of 228 heuristics in order to adapt the tool to the evaluation of different industrial requirements. Finally, the set of new heuristics has been validated by experts using the System Usability Scale (SUS) questionnaire and the categories has been prioritized in order of their importance in the evaluation of Human-Robot Interaction through the Analytic Hierarchy Process (AHP).

Each step time-limited iterative learning control for a cushion robot with motion velocity constraints

An each step time-limited iterative learning control model was developed for a cushion robot with velocity constraints. A predictive modeling method was proposed to solve the velocity mutation problem by employing a kinematic model to constrain the velocity inputs, which can then constrain the robot’s actual velocity. Furthermore, a tracking error system was established that used constrained motion velocity and a dynamic model. A new iterative controller with each step time-limited learning was built to reduce the transient adjustment time. Simulation results and comparative analyses revealed that the proposed control method quickly stabilizes the system and ensures that the human–robot system operates at a safe velocity.

Nonlinear-Observer-Based Neural Fault-Tolerant Control for a Rehabilitation Exoskeleton Joint with Electro-Hydraulic Actuator and Error Constraint

The rehabilitation exoskeleton is an effective piece of equipment for stroke patients and the aged. However, this complex human–robot system incurs many problems, such as modeling uncertainties, unknown human–robot interaction, external disturbance, and actuator fault. This paper addresses the adaptive fault-tolerant tracking control for a lower limb rehabilitation exoskeleton joint driven by an electro-hydraulic actuator (EHA). First, the model of the exoskeleton joint is built by considering the principle of the hydraulic cylinder and the servo valve. Then, a novel disturbance-observer-based neural fault-tolerant control scheme is proposed, where the neural network and disturbance observer are incorporated to reduce the influence of the the nonlinear uncertainties and disturbance. Meanwhile, a barrier Lyapunov function is constructed to ensure the stability of the closed-loop system. Finally, comparative simulations on an exoskeleton joint validate the effect of the proposed control scheme.

Generating Function Reallocation to Handle Contingencies in Human–Robot Teaming Missions: The Cases in Lunar Surface Transportation

On lunar missions, efficient and safe transportation of human–robot systems is essential for the success of human exploration and scientific endeavors. Given the fact that transportation constructs bottlenecks for numerous typical lunar missions, it is appealing to investigate what function allocation strategies can generate optimal task implementation paths for robots with low-human workloads when the situation changes. Thus, this paper presents a novel approach to dynamic human–robot function allocation explicitly designed for team transportation in lunar missions. The proposed dynamic allocation framework aims to optimize human–robot collaboration by responding to existing and potential contingencies. First, a fitness concept model is designed to quantify the factors that motivate the functional adaptation of each agent in dynamic lunar mission scenarios. A hierarchical reinforcement learning (HRL) algorithm with two layers is then employed for decision-making and optimization of human–robot function allocation. Finally, the validity of the framework and algorithm proposed is validated by a series of human–robot function allocation experiments on a simulated environment that mimics lunar transportation scenarios, and is compared with the performance of other algorithms. In the future, path-planning algorithms can be incorporated into the proposed framework to improve the adaptability and efficiency of the human–robot function allocation in lunar missions.

Spatial Repetitive Impedance Learning Control for Robot-Assisted Rehabilitation

In robot-assisted rehabilitation and leg exoskeletons, humans and robots are required to collaboratively complete repetitive tasks with fixed periodic paths. In such applications, impedance learning control can provide variable impedance regulation for improving the performance of physical interactions; however, designing such control is highly challenging owing to the difficulty in modeling human time-varying dynamics. By exploiting the spatial periodicity characteristics of the desired trajectory and human impedance, we propose a novel spatial repetitive impedance learning control strategy to enhance interaction performance. First, a defined spatial operator serves as the mathematics foundation for constructing the robot dynamics in the spatial domain. Then, a spatial impedance learning controller is designed. In this article, time-varying impedance profiles are estimated using spatial full-saturation iterative learning laws, while robotic parameter uncertainties are estimated using the differential adaptation law with projection modification. We validate the uniform convergence of the tracking error through a Lyapunov-like analysis and demonstrate the control effectiveness using an illustrative example. Compared with related results on temporal repetitive learning control, the proposed control approach can enable human–robot system to complete a repetitive task with unspecified speeds according to the users' strengths and motion capacity.

Simulation, Learning, and Application of Vision-Based Tactile Sensing at Large Scale

Large-scale robotic skin with tactile sensing ability is emerging with the potential for use in close-contact human–robot systems. Although recent developments in vision-based tactile sensing and related learning methods are promising, they have been mostly designed for small-scale use, such as by fingers and hands, in manipulation tasks. Moreover, learning perception for such tactile devices demands a huge tactile dataset, which complicates the data collection process. To address this, this study introduces a multiphysics simulation pipeline, called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SimTacLS</i> , which considers not only the mechanical properties of external physical contact but also the realistic rendering of tactile images in a simulation environment. The system utilizes the obtained simulation dataset, including virtual images and skin deformation, to train a tactile deep neural network to extract high-level tactile information. Moreover, we adopt a generative network to minimize sim2real inaccuracy, preserving the simulation-based tactile sensing performance. Last but not least, we showcase this sim2real sensing method for our large-scale tactile sensor ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TacLink</i> ) by demonstrating its use in two trial cases, namely, whole-arm nonprehensile manipulation and intuitive motion guidance, using a custom-built tactile robot arm integrated with TacLink. This article opens new possibilities in the learning of transferable tactile-driven robotics tasks from virtual worlds to actual scenarios without compromising accuracy.

Training Path Optimization Method of a Moveable Multifunction Rehabilitation Robot

Passive path tracking training is usually used for robot assisted lower limb rehabilitation. The movement of robot will cause the fluctuation of joint force. This letter presents an optimal training path planning algorithm to reduce the patient's joint force caused by a moveable multifunction rehabilitation robot during passive pedaling training. The dynamics model of the human-robot system is established, and the joint forces of the patient are analysed. The planning process is divided into a series of motion steps based on the original training path, at which the optimal algorithm is applied to search the best path. Then, the optimal path is fitted by Bezier curve to get a closed training trajectory. Finally, a Bionic Experiment Platform (BEP) and Human-robot Interaction Platform (HIP) are established to verify the effectiveness of the optimal algorithm. The (BEP) experimental results show that the knee joint force drops by an average of 12.71%, and the maximum drops by 15.41%.The HIP experiment results show that the interaction force drops by an average of 8.5%.

Comparison of the dynamics of exoskeletal-assisted and unassisted locomotion in an FDA-approved lower extremity device: Controlled experiments and development of a subject-specific virtual simulator.

Robotic exoskeletons have considerable, but largely untapped, potential to restore mobility in individuals with neurological disorders, and other conditions that result in partial or complete immobilization. The growing demand for these devices necessitates the development of technology to characterize the human-robot system during exoskeletal-assisted locomotion (EAL) and accelerate robot design refinements. The goal of this study was to combine controlled experiments with computational modeling to build a virtual simulator of EAL. The first objective was to acquire a minimum empirical dataset comprising human-robot kinematics, ground reaction forces, and electromyography during exoskeletal-assisted and unassisted locomotion from an able-bodied participant. The second objective was to quantify the dynamics of the human-robot system using a subject-specific virtual simulator reproducing EAL compared to the dynamics of normal gait. We trained an able-bodied participant to ambulate independently in a Food and Drug Administration-approved exoskeleton, the ReWalk P6.0 (ReWalk Robotics, Yoknaem, Israel). We analyzed the motion of the participant during exoskeletal-assisted and unassisted walking, sit-to-stand, and stand-to-sit maneuvers, with simultaneous measurements of (i) three-dimensional marker trajectories, (ii) ground reaction forces, (iii) electromyography, and (iv) exoskeleton encoder data. We created a virtual simulator in OpenSim, comprising a whole-body musculoskeletal model and a full-scale exoskeleton model, to determine the joint kinematics and moments during exoskeletal-assisted and unassisted maneuvers. Mean peak knee flexion angles of the human subject during exoskeletal-assisted walking were 50.1° ± 0.6° (left) and 52.6° ± 0.7° (right), compared to 68.6° ± 0.3° (left) and 70.7° ± 1.1° (right) during unassisted walking. Mean peak knee extension moments during exoskeletal-assisted walking were 0.10 ± 0.10 Nm/kg (left) and 0.22 ± 0.11 Nm/kg (right), compared to 0.64 ± 0.07 Nm/kg (left) and 0.73 ± 0.10 Nm/kg (right) during unassisted walking. This work provides a foundation for parametric studies to characterize the effects of human and robot design variables, and predictive modeling to optimize human-robot interaction during EAL.

Drivable Space of Rehabilitation Robot for Physical Human–Robot Interaction: Definition and an Expanding Method

Physical human–robot interaction performance of present rehabilitation robots are still not satisfactory in the clinical practice. Especially, the work space where the robot can be driven smoothly by users is still very limited, which prevents rehabilitation robots from being applied successfully. In this study, a new concept of drivable space is proposed to evaluate the work spaces of rehabilitation robots, and a method for expanding the drivable space is designed based on the dynamics of the coupled human–robot system and human joint characteristics. First, the definition of drivable space is presented based on comparison of human joint torques, and the minimal torques necessary to drive robot joints, which is mainly determined by the torque estimation errors for general rehabilitation robots driven smoothly by motors. Therefore, a method for improving torque estimation accuracies based on dynamics modeling is then designed. A data-driven error prediction method based on Gaussian process regression is proposed to adaptively compensate the model errors, by which the most accurate dynamic model so far for the coupled system can be obtained, and a method for generation of the training dataset, which is used in error prediction, is designed as well. Moreover, the torque–angle relationship of human joints is modeled and used to optimize the torque error distribution, by which it can be proven that the drivable space can be further expanded. Finally, performance of the proposed methods are demonstrated and validated by experiments carried out on a lower limb rehabilitation robot.

Model Predictive Degree of Automation Regulation for Mobile Robots Using Robot Vitals and Robot Health

Environmental adversities can severely impact the performance of human-robot teams, potentially even leading to task failure. If the operator and the robot automation are not equally affected, adjusting the degree of automation to shift control authority between them is a means of maintaining the performance of the human-robot team. The robot vitals and robot health framework is a recent approach to quantifying runtime performance degradation in robots. This framework can serve as a methodological foundation for the adjustment of the degree of automation based on the human-robot system's state. In this paper, we contribute two model predictive adaptive automation systems that can adjust either the level or the degree of automation of a robot. These systems optimize robot health to ensure optimal performance of the human-robot team when exposed to adversities. Feasibility studies in simulation showcase the ability of our systems to manage the level and degree of automation, thus allowing for an optimal task execution by the human-robot team.