Safe Human-robot Collaboration Research Articles

Unleashing mixed-reality capability in Deep Reinforcement Learning-based robot motion generation towards safe human–robot collaboration

The integration of human–robot collaboration yields substantial benefits, particularly in terms of enhancing flexibility and efficiency within a range of mass-personalized manufacturing tasks, for example, small-batch customized product inspection and assembly/disassembly. Meanwhile, as human–robot collaboration lands broader in manufacturing, the unstructured scene and operator uncertainties are increasingly involved and considered. Consequently, it becomes imperative for robots to execute in a safe and adaptive manner rather than solely relying on pre-programmed instructions. To tackle it, a systematic solution for safe robot motion generation in human–robot collaborative activities is proposed, leveraging mixed-reality technologies and Deep Reinforcement Learning. This solution covers the entire process of collaboration starting with an intuitive interface that facilitates bare-hand task command transmission and scene coordinate transformation before the collaboration begins. In particular, mixed-reality devices are employed as effective tools for representing the state of humans, robots, and scenes. This enables the learning of an end-to-end Deep Reinforcement Learning policy that addresses both the uncertainties in robot perception and decision-making in an integrated manner. The proposed solution also implements policy simulation-to-reality deployment, along with motion preview and collision detection mechanisms, to ensure safe robot motion execution. It is hoped that this work could inspire further research in human–robot collaboration to unleash and exploit the powerful capabilities of mixed reality.

Modeling of flexible shaft for robotics applications

Reducing moving mass and effective inertia is essential for achieving safe human–robot collaboration. This can be achieved either by employing remote actuation, which moves the mass of actuators away from the moving elements of the robot, or by elastic actuation, which decouples the inertia of the actuator from the inertia of the robot’s link. Flexible shafts, being torsionally compliant slender long shafts, offer a combination of remote and series elastic actuation over an obstacle. Modeling such transmission is complicated due to its three-dimensional deformation in torsion, bending, and helical buckling. This paper proposes an algorithm for estimating the polar moment of inertia and stiffness of the flexible shafts using their physical dimensions of length and diameter. Using an inertia-spring–damper model, the stiffness parameters are estimated experimentally for nine flexible shafts with variable diameters and lengths in straight and bent configurations. The proposed algorithm is validated with experimental stiffness values for variable diameters and lengths in straight configuration. For bent configuration, an empirical formulation is provided to incorporate the bending deformation effect till a bending angle of 45∘.

Fusion of Time-of-Flight Based Sensors with Monocular Cameras for a Robotic Person Follower

Human-robot collaboration (HRC) is becoming increasingly important in advanced production systems, such as those used in industries and agriculture. This type of collaboration can contribute to productivity increase by reducing physical strain on humans, which can lead to reduced injuries and improved morale. One crucial aspect of HRC is the ability of the robot to follow a specific human operator safely. To address this challenge, a novel methodology is proposed that employs monocular vision and ultra-wideband (UWB) transceivers to determine the relative position of a human target with respect to the robot. UWB transceivers are capable of tracking humans with UWB transceivers but exhibit a significant angular error. To reduce this error, monocular cameras with Deep Learning object detection are used to detect humans. The reduction in angular error is achieved through sensor fusion, combining the outputs of both sensors using a histogram-based filter. This filter projects and intersects the measurements from both sources onto a 2D grid. By combining UWB and monocular vision, a remarkable 66.67% reduction in angular error compared to UWB localization alone is achieved. This approach demonstrates an average processing time of 0.0183s and an average localization error of 0.14 meters when tracking a person walking at an average speed of 0.21 m/s. This novel algorithm holds promise for enabling efficient and safe human-robot collaboration, providing a valuable contribution to the field of robotics.

An Integrated Architecture for Robotic Assembly and Inspection of a Composite Fuselage Panel with an Industry 5.0 Perspective

Aeronautical robotic applications use quite large, heavy robots with huge end effectors that are frequently multifunctional. An assembly jig to hold a fuselage panel and two medium-sized six-axis robots fixed on linear axes, referred to as the internal and the external robot with respect to the curvature of the panel, make up the Lean robotized AssemBly and cOntrol of composite aeRostructures (LABOR) work cell. A distributed software architecture is proposed in which individual modules are developed to execute specific subprocesses, each implementing innovative algorithms that solve the main drawbacks of state-of-the-art solutions. Real-time referencing adopts a point-cloud-based strategy to reconstruct and process the part before drilling, avoiding hole positioning errors. Accurate concentric countersink diameters are made possible through the automatic adjustment of the drilling tool with respect to the skin panel, which guarantees its orthogonality, as well as the implementation of process parameter optimization algorithms based on historical results that compensate for the wear of the drilling bits. Automatic sealing and fastening strategies that involve the measurement of the main fastener quality parameters allow for the complete verification of the entire assembly process of each part. Additionally, an advanced multimodal perception system continuously monitors the collaborative workspace to ensure safe human–robot collaboration (HRC) tasks. Through this integrated architecture, LABOR substantially reduces expenses and facilitates maintenance and programming.

TacSuit: A Wearable Large-Area, Bioinspired Multimodal Tactile Skin for Collaborative Robots

Robots are now working more and more closely with humans outside of traditional fences in industrial scenes. Their real-time tactile interaction perception is crucial to the safety of human–robot collaboration (HRC). In this work, we present a customized, wearable and modular robot skin (TacSuit), which is scalable for large-area surface coverage of robot with easily accessible multi-modal sensors, including pressure, proximity, acceleration and temperature sensors. The TacSuit is co-designed for mechanical structure and data fusion algorithm, consisting of three levels of design: sensor, cell (of multi-modal sensors) and block (of multiple cells). These sensors are stored with custom-designed and 3D printed capsules to achieve the conformity, scalability and easy installation to the arbitrary robot surface. A multi-level event-driven data fusion algorithm enables efficient information processing for large number of tactile sensors. Furthermore, a virtual interaction force fusion method takes both the proximity and force perception information into consideration in order to achieve safety of whole interaction process before and after direct physical contacts. A humanoid robotic platform successfully realizes the TacSuit wear of 159 tactile cells. Validation experiments of obstacle detection demonstrate the effective collision avoidance capability of the TacSuit for safe HRC.

Task reallocation of human-robot collaborative production workshop based on a dynamic human fatigue model

As collaborative robots become more popular in industry, they are used to collaborate with human workers and free human workers from repeated and heavy work and reduce the risk of physical injuries. However, human workers need to finish tasks that require high cognitive abilities, which increases the risk of mental fatigue. Fatigue can negatively impact workers and even threaten the safety of workers. Currently, the dynamics of human fatigue model are rarely considered. Hence, it is crucial to develop a dynamic allocation method for human-robot collaborative tasks that considers dynamic human fatigue to ensure efficient and safe human-robot collaboration. This paper proposes a task reallocation method for human-robot collaborative production workshop based on a dynamic human fatigue model. A fatigue feature extraction method from multimodal data and a fatigue evaluation network are proposed to evaluate the current fatigue status of human workers. Based on the evaluation results, the human fatigue model is dynamically updated. If the worker’s fatigue state changes, a task reallocation scheme is generated by an improved genetic algorithm based on reinforcement learning and the dynamic human fatigue model. The experimental results show that the proposed model and algorithm are effective and efficient, and can provide innovative insights for realizing a human-centered manufacturing workshop.

Real-time constraint-based planning and control of robotic manipulators for safe human–robot collaboration

A recent trend in industrial robotics is to have robotic manipulators working side-by-side with human operators. A challenging aspect of this coexistence is that the robot is required to reliably solve complex path-planning problems in a dynamically changing environment. To ensure the safety of the human operator while simultaneously achieving efficient task realization, this paper introduces a computationally efficient planning and control architecture that combines a Rapidly-exploring Random Tree (RRT) path planner with a trajectory-based Explicit Reference Governor (ERG) by means of a reference selector. The resulting scheme can steer the robot arm to the desired end-effector pose in the presence of actuator saturation, limited joint ranges, speed limits, a cluttered static obstacle environment, and moving human collaborators. The effectiveness of the proposed framework is experimentally validated on the Franka Emika Panda robot arm and fed with feedback information from state-of-the-art depth perception systems. Our method outperforms both the standalone RRT and ERG algorithms in cluttered static environments where it overcomes: i) the RRT’s inability to handle dynamic constraints which result in constraint violations and ii) the ERG’s undesirable property of getting trapped in local minima. Finally we employed the RRT+ERG in highly dynamic human–robot coexistence experiments without sacrificing the real-time requirements.

Human tracking from quantised sensors: An application to safe human–robot collaboration

The proliferation of cage-less robotic applications is justifying this research which proposes a method to process the output of safety sensors with the aim of maximising the productivity of the robot in a collaborative scenario. Particularly, the Speed and Separation Monitoring (SSM) strategy, which prescribes the robot to reduce its speed proportionally to the vicinity of the human, will be investigated. In state-of-the-art industrial implementations, SSM is implemented in a very conservative way, without exploiting the capabilities of modern sensing devices. This work proposes a methodology to improve the performance of SSM algorithms while dealing finite and quantised 2D cost-effective sensing capabilities. The strategy is verified experimentally as applied on a palletising application with a ComauSmartSix industrial robot, showing slightly improved performance with respect to standard practice.

Safe human-robot collaboration in construction: A conceptual perspective

Introduction: Small mobile robots have become increasingly popular in the construction domain over the last few years. They are stable on rough terrains, can walk over small obstacles, climb stairs, and carry various sensors or arms to perform diverse functions and sub-tasks required to complete construction-related tasks. Saving time, improving accessibility to difficult or unsafe spaces, and reducing costs while accomplishing construction tasks are some of the benefits of using small, mobile robots in construction. However, serious concerns about new workplace hazards could arise from having mobile robots on the jobsite. Unfortunately, no study has attempted to evaluate these risks, especially in the construction domain. Therefore, there was a significant need to develop a holistic understanding of the direct and indirect risks of mobile robot applications in construction. Method: In this paper, we used inferential and Virtual Reality (VR) visualization techniques to: (1) construct conceptual visualizations of proximal and distant human-robot interaction within the construction context; and (2) identify potential safety challenges of robots, which were categorized into three groups: (a) physical risks, (b) attentional costs, and (c) psychological impacts. These identified safety challenges were then validated and ranked by a group of construction safety and robotic experts who had knowledge and experience using such robots in construction. Practical Applications: The outcomes of the study provided a detailed understanding of how robots might adversely affect workers’ safety and health. The study outcomes could also be ultimately used in creating regulatory and administrative guidelines for the safe operations of small mobile robots in construction.

Safe human–robot collaboration for industrial settings: a survey

Safe human–robot collaboration for industrial settings: a survey

Safe Human-Robot Collaboration via Collision Checking and Explicit Representation of Danger Zones

Safe Human-Robot Collaboration via Collision Checking and Explicit Representation of Danger Zones

Towards a Safe Human-Robot Collaboration Using Information on Human Worker Activity.

Most industrial workplaces involving robots and other apparatus operate behind the fences to remove defects, hazards, or casualties. Recent advancements in machine learning can enable robots to co-operate with human co-workers while retaining safety, flexibility, and robustness. This article focuses on the computation model, which provides a collaborative environment through intuitive and adaptive human-robot interaction (HRI). In essence, one layer of the model can be expressed as a set of useful information utilized by an intelligent agent. Within this construction, a vision-sensing modality can be broken down into multiple layers. The authors propose a human-skeleton-based trainable model for the recognition of spatiotemporal human worker activity using LSTM networks, which can achieve a training accuracy of 91.365%, based on the InHARD dataset. Together with the training results, results related to aspects of the simulation environment and future improvements of the system are discussed. By combining human worker upper body positions with actions, the perceptual potential of the system is increased, and human-robot collaboration becomes context-aware. Based on the acquired information, the intelligent agent gains the ability to adapt its behavior according to its dynamic and stochastic surroundings.

Identification of Contact Avoidance Zones of Robotic Devices in Human-Robot Collaborative Workspaces

The objective of this study was to establish a framework for producing indoor maps and locating robotic devices in a manufacturing environment. The framework employs computer vision techniques to construct the map and identify the presence of human workers. It defines the contact avoidance zones around human workers and existing obstacles. Once the location of the robot is identified, the map is used to plan paths to ensure safe human-robot collaboration for mobile and collaborative robots in shared workspaces with humans. The incorporation of avoidance zones into the map allows the robotic devices to anticipate the movements of workers and prevent collisions, this decreases the risk of injuries in collaborative environments. This paper illustrates the implementation of robots evading unforeseen contact with pre-defined contact avoidance zones, employing two distinct examples as demonstration

A Learning Based Hierarchical Control Framework for Human–Robot Collaboration

In this paper, using the ball and beam system as an illustration, a control scheme is developed on human-robot collaboration, i.e., a two-level hierarchical framework is proposed to establish a robust human-robot collaboration (HRC) policy. On the high level, a deep reinforcement learning (DRL) algorithm is presented to plan the desired beam rotational velocity. The low level is constructed by a human-intention perception module and a robust collaboration policy design module. For the first module, a probabilistic model is fitted by using the Gaussian process regression (GPR) approach to predict human-hand velocities, and prediction results follow Gaussian distributions where mean values and variances represent predicted human-hand velocities and corresponding prediction confidences, respectively. For the second module, a robust collaboration policy is established by fusing a proactive policy and a conservative policy, where the proactive policy is used to control the robot to achieve the desired beam rotational velocity by using the predicted human-hand velocities. The conservative policy is designed to ensure the collaboration safety. The weighted parameters for fusion are adaptively tuned based on the prediction precision and confidence. Experiments are conducted on controlling ball position on a beam jointly by a human and a robot with vision data, and experimental results show the effectiveness of the designed robust collaboration policy. Note to Practitioners—Predicting human future behaviors and moderating robot behaviors accordingly is a long-standing problem for human-robot collaboration (HRC) tasks, such as assembling, transporting, etc. Existing approaches generally regard human behaviors as noises or only build simple human models without prediction confidence. This paper proposes a learning-based hierarchical framework that will derive a robust and safe HRC policy considering human behaviors, prediction confidence, and task-related optimality. The framework is validated by a representative experiment where human and robot are asked to jointly control a ball and beam system.

A Connective Framework for Safe Human–Robot Collaboration in Cyber-Physical Production Systems

A Connective Framework for Safe Human–Robot Collaboration in Cyber-Physical Production Systems

NARX Neural Network for Safe Human–Robot Collaboration Using Only Joint Position Sensor

Background: Safety is the very necessary issue that must be considered during human-robot collaboration in the same workspace or area. Methods: In this manuscript, a nonlinear autoregressive model with an exog-enous inputs neural network (NARXNN) is developed for the detection of collisions between a manipulator and human. The design of the NARXNN depends on the dynamics of the manipulator’s joints and considers only the signals of the position sensors that are intrinsic to the manipulator’s joints. Therefore, this network could be applied and used with any conventional robot. The data used for training the designed NARXNN are generated by two experiments considering the sinusoidal joint motion of the manipulator. The first experiment is executed using a free-of-contact motion, whereas in the second experiment, random collisions by human hands are performed with the robot. The training process of the NARXNN is carried out using the Levenberg–Marquardt algorithm in MATLAB. The evaluation and the effectiveness (%) of the developed method are investigated taking into account different data and conditions from the used data for training. The experiments are executed using the KUKA LWR IV manipulator. Results: The results prove that the trained method is efficient in estimating the external joint torque and in correctly detecting the collisions. Conclusions: Eventually, a comparison is presented between the proposed NARXNN and the other NN architectures presented in our previous work.

Safety and Efficiency in Robotics: The Control Barrier Functions Approach

This article aims at presenting an introductory overview of the theoretical framework of control barrier functions (CBFs) and of their application to the design of safety-related controllers for robotic systems. The article starts by describing the basic concepts of CBFs and how they can be used to build optimization problems embedding CBF-based constraints, whose solutions correspond to the control input enforcing the desired safety properties into the behavior of the controlled system. Simple examples, understandable for readers with a basic background in control theory and accompanied with source code for their simulation, are used to highlight the appealing features of the CBF-based approach. Then, more complex formulations applicable to robotic manipulators are introduced, recalling recent literature results allowing users to implement a robust design methodology. Finally, it is shown that CBFs are suitable for the implementation of safe human–robot collaboration in a realistic industrial scenario by means of the experimental validation in an industrial setup for collaborative robotics.

Human-centric robotic manipulation in construction: generative adversarial networks based physiological computing mechanism to enable robots to perceive workers’ cognitive load

With the recent advancements in sensing technologies, mechatronics, and artificial intelligence, collaborative robots are deployed on construction sites to assist workers in performing physically demanding tasks. However, the human–robot collaboration (HRC) can bring several occupational challenges to workers, ranging from physical collisions to adverse psychological impacts. To date, most of the literature on HRC has focused on addressing physical safety challenges, while very few have considered the psychological safety of the workers. To bridge this gap, by integrating generative adversarial network, autoencoder, machine learning, and robot adaptation techniques, this study proposes a novel physiological computing system that enables the collaborative robot to efficiently perceive workers’ psychological states and regulate its performance seamlessly. The results showed that the proposed system allowed the robot to adjust its performance as per workers’ cognitive load level with 89.6% accuracy. The findings revealed the potential of the proposed system in facilitating safe HRC in construction.

FESTO UNVEILS FIRST PNEUMATIC COBOT

The world’s first pneumatic cobot has been revealed by Festo, heralding a new era in safe human-robot collaboration.

Design, Modeling and Validation of a Tendon-Driven Soft Continuum Robot for Planar Motion Based on Variable Stiffness Structures

Robotic structures based on variable stiffness enable high-performance and flexible motion systems that are inherently safe and thus allow safe Human-Robot Collaboration. This letter presents the design of a robotic structure based on variable stiffness. A robotic manipulator is developed using three variable stiff segments based on particle jamming with a backbone architecture and two tendons for an underactuated motion control of the whole structure. By switching the structural stiffness, the manipulator is able to perform complex planar motion with only one pair of tendons, reducing the number of actuators required. A kinematic modeling approach for the calculation of the forward kinematics of this soft continuum structure is presented, and the validation on the real system is explained. The kinematic simulation is performed with a multibody simulation model (MBS) using rigid body elements in combination with rotational springs. The validation of the model is carried out with visual measurements of the real system using defined target shapes. Simulation and experimental results are discussed and compared also with a common constant curvature model. The developed MBS-model demonstrates a promising modeling approach with a position error lower 3% for the calculation of the presented manipulator under gravity.