Human-robot Collaboration Systems Research Articles

Modelling Uncertainties in Human-Robot Industrial Collaborations

With the rise of Industry 4.0 technological trends, there is a growing tendency in manufacturing automation towards collaborative robots. Human-robot collaboration (HRC) is motivated by the combination of complementary human and robot skills and intelligence, which can increase productivity, flexibility and adaptability. However, it is still challenging to achieve safe and efficient human-robot collaborative systems due to the dynamics of human presence, uncertainties in the dynamic environment, and the need for adaptability. Such uncertainties could relate to the human-robot capabilities and availability, parts positioning, unexpected obstacles, etc. This paper develops time-based simulations and event-based simulations to model and analyse the dynamic factors in human-robot collaboration systems. The novelty of this work is the systematic modelling and analysis of dynamic factors in HRC manufacturing scenarios through the development of digital simulations of human-robot collaboration scenarios while considering the dynamic nature of humans and environments. A real-world industrial case study was redesigned into a collaborative workstation. The simulated scenario is developed using the software called Tecnomatix Process Simulate, which can help to visualise the dynamic factors and analyse the impact of the factors on the HRC. The simulation illustrates and analyses possible uncertainties in human-robot industrial collaborative workstations, which can contribute to the future design of HRC industrial workstations and the optimisation of productivity.

The influence of collision avoidance strategies on human-robot collaborative systems

Collaborative robots improve the traditional production systems, both automatic and manual, by allowing the resources to share the workspace. However, the shared workspace is both the greatest advantage and limit of collaborative robots. Indeed, interference between the human operator and the cobot may result in an emergency stop, reducing the performance of the system. Hence, a collision avoidance strategy is required to avoid collision; however, this may affect the performance of the system. Therefore, it is necessary to investigate how the shared workspace affects the system performance, and the effects of the adoption of a collision avoidance strategy. To this regard, a collision avoidance strategy has been developed and it is presented in this work and an experimental validation proved the efficacy of the proposed strategy. Lastly, simulation tests were carried out to analyse the performance of the system for different sizes of the collaboration area with respect to the workspace. The results shows that, while the increase in the collaboration area reduces the performance of the system, collision avoidance strategies mitigate this effect.

Exploring the operator’s perspective within changeable and automated manufacturing – A literature review

The current industrial environment faces an increased demand for products to be delivered to market at a higher pace with high variety and customization. An increasing trend for automation and changeability to manage the increased volume of variants further intensify the challenge to design and integrate a manufacturing process that effectively incorporates the capacity to fulfil both aspects. The concept of reconfigurable manufacturing system (RMS) is a system designed to rapidly react to market opportunities and changes while enabling adaptability on a production system and machine level. RMS provides sufficient manufacturing flexibility in combination with dedicated manufacturing that strikes an effective balance between the two. RMS can utilize emerging technologies such as Automation, Integrated Internet of Things (IIoT), Cloud Computing, and Simulation to increase its ability to manage flexibility. Scalability can be achieved with automated manufacturing solutions such as human-robot collaboration (HRC). The operator, in these RMS that utilize emerging technologies, is an integral part of a manufacturing system, due to the interactivity between operators and reconfigurable or automated equipment. Extensive research has been conducted on technical solutions, safety challenges, ergonomics, and interfaces of the technology within HRC systems, yet limited research has been conducted on such systems from the operator’s perspective. The aim of this article is to explore current academic literature on the topic of HRC and analyse its relevance within changeability as well as identify potential knowledge gaps of implementing an HRC system within RMS from an operator’s perspective.

Challenges in human-robot collaborative assembly in shipbuilding and ship maintenance, repair and conversion (SMRC) industry

In recent years research in manufacturing has been shifting towards flexible and intuitive Human-Robot Collaborative (HRC) solutions, allowing operators to coexist and interact safely with robots of various payloads. Previous research provided proof that HRC systems can improve the quality and cycle time of processes, including welding, grinding, polishing, assembly and picking. Applying such solutions to shipbuilding and ship maintenance, repair and conversion (SMRC) industry though is not straight forward. Lack of 3D-CAD models, variety of processes that have to be performed, manipulation of dissimilar parts, confined spaces, unpredictable external conditions are some of the challenges that arise.Nevertheless, the shipbuilding and SMRC industry is a dynamic and competitive sector, which could highly benefit from leveraging technology advancements, emerged from Industry 4.0 revolution. These advancements can not only improve the work life of the operators, supporting them in dangerous working environments, but also increase the quality of the final product as well as productivity rate.In this direction, this paper aims to present the challenges that derive from the shipbuilding and SMRC industry related with assembly operations, as well as a novel user-centric solution that aspires to overcome these challenges. The solution consists of a reconfigurable HRC cell, involving a high payload collaborative manipulator able to perform lifting and positioning of large heavy parts, while the operator performs the welding task. Intuitive interaction solutions allow precise parts handling by workers. Moreover, multi-layer safety systems allow the monitoring of the workplace and ensure the safety of the operators. Finally, hand guiding, and intuitive offline programming techniques enable the manipulation and programming of the high-payload robot by not experts.

Impact of VR Technology on a Human in Semi-autonomous Multi-robot Navigation: Control-Theoretic Perspective

In this paper, we investigate a scenario of one-human-multiple-robot navigation in three dimensions, and examine the impacts of the VR (Virtual Reality) technology on human properties from a control-theoretic perspective. We start by reviewing a passivity-based distributed control architecture that takes complementary interactions such that motion synchronization is autonomously completed by a distributed robot controller while the operator is dedicated to robot navigation. Due to the limited human capability of 3-D recognition and limited dimensionality on the real-time manipulability, 3-D navigation is completely different from that of the one- or two-dimensional case and we need to carefully design both feedback and command interfaces between the operator and robots. Specifically, we employ two different pairs of the interfaces, traditional joystick controller with 2-D display monitor and HMD (Head Mounted Display) with VR controller. We then build human models from the operation data with these interfaces on a human-in-the-loop simulator. Through the human modeling, we present two novel findings: (i) VR interfaces improve the accuracy of the human model with about 25∼45% of fitting ratio, which must drastically eases the design of human-robot collaboration systems, (ii) VR interfaces enhance human passivity, which is a key to ensuring closed-loop stability for the human-in-the-loop system.

A visual reasoning-based approach for mutual-cognitive human-robot collaboration

Human-robot collaboration (HRC) allows seamless communication and collaboration between humans and robots to fulfil flexible manufacturing tasks in a shared workspace. Nevertheless, existing HRC systems lack an efficient integration of robotic and human cognitions. Empowered by advanced cognitive computing, this paper proposes a visual reasoning-based approach for mutual-cognitive HRC. Firstly, a domain-specific HRC knowledge graph is established. Next, the holistic manufacturing scene is perceived by visual sensors as a temporal graph. Then, a collaborative mode with similar instructions can be inferred by graph embedding. Lastly, mutual-cognitive decisions are immersed into the Augmented Reality execution loop for intuitive HRC support.

Dynamic Scene Graph for Mutual-Cognition Generation in Proactive Human-Robot Collaboration

Human-robot collaboration (HRC) plays a crucial role in agile, flexible, and human-centric manufacturing towards the mass personalization transition. Nevertheless, in today’s HRC tasks, either humans or robots need to follow the partners’ commands and instructions along collaborative activities progressing, instead of proactive, mutual engagement. The non-semantic perception of HRC scenarios impedes mutually needed, proactive planning and high-cognitive capabilities in existing HRC systems. To overcome the bottleneck, this research explores a dynamic scene graph-based method for mutual-cognition generation in Proactive HRC applications. Firstly, a spatial-attention object detector is utilized to dynamically perceive objects in industrial settings. Secondly, a linking prediction module is leveraged to construct HRC scene graphs. An attentional graph convolutional network (GCN) is utilized to capture relations between industrial parts, human operators, and robot operations and reason structural connections of human-robot collaborative processing as graph embedding, which links to mutual planners for human operation supports and robot proactive instructions. Lastly, the Proactive HRC implementation is demonstrated on disassembly tasks of aging electronic vehicle batteries (EVBs) and evaluate its mutual-cognition capabilities.

Human-robot collaboration empowered by hidden semi-Markov model for operator behaviour prediction in a smart assembly system

Human-robot collaboration (HRC) in assembly scenarios, particularly a human center HRC system, demands the robot to react smartly. How the robot should respond in time to collaboration requests with anticipation of human behavior instead of waiting for visible cues is challenging. This study proposes a hidden semi-Markov model (HSMM) for a human center HRC assembly system that enables the robot to accurately predict the human assembly rate. The HSMM model links different assembly rates (visible states) with distinct operator perceptions (invisible states). Thus, the robot responds by adjusting to the predicted human behavior using the proposed HSMM. The proposed model was verified through a simulation of the collaborative scenario on a graphics processing unit assembly line. The results indicate that the proposed HSMM-based HRC plan significantly reduces the assembly cycle time and operator waiting time. A successful partnership between humans and robots in a smart assembly system using the proposed recognition intelligence and precise predictions was achieved in this study.

Biomechanical Effects on Lower Extremities in Human-Robot Collaborative Agricultural Tasks

The present study pertains to a key aspect of human-robot collaborative systems which is usually underestimated, namely occupational health prolepsis. The aim of this investigation was to assess the biomechanical effects of manual symmetric load lifting related to a synergistic agricultural task that utilizes an unmanned ground vehicle to undertake the carriage of loads. Towards that goal, kinetic and kinematic data were collected from the lower extremities of thirteen experienced workers, by testing three different deposit heights (70, 80, 90 cm) corresponding to possible adjustments of the available agricultural robot. Moreover, the muscle activation levels of three lower extremity muscles and one trunk muscle were evaluated via a wireless electromyography system. Overall, the experimental findings revealed that the lower examined load height was associated with larger knee flexion moments and hip extension moments. Nevertheless, this height was related to lower activation mainly of the erectus spinae muscles. Finally, insignificant alterations were observed for the ankle joint as well as the activation levels of the other muscles. Consequently, a height equal to 90 cm is suggested, however, by avoiding extreme lumbar postures. The current results can be exploited for possible ergonomic interventions concerning the optimal deposit height of a robotic platform when a similar case is designed.

Digital Twin-Driven Human Robot Collaboration Using a Digital Human.

Advances are being made in applying digital twin (DT) and human–robot collaboration (HRC) to industrial fields for safe, effective, and flexible manufacturing. Using a DT for human modeling and simulation enables ergonomic assessment during working. In this study, a DT-driven HRC system was developed that measures the motions of a worker and simulates the working progress and physical load based on digital human (DH) technology. The proposed system contains virtual robot, DH, and production management modules that are integrated seamlessly via wireless communication. The virtual robot module contains the robot operating system and enables real-time control of the robot based on simulations in a virtual environment. The DH module measures and simulates the worker’s motion, behavior, and physical load. The production management module performs dynamic scheduling based on the predicted working progress under ergonomic constraints. The proposed system was applied to a parts-picking scenario, and its effectiveness was evaluated in terms of work monitoring, progress prediction, dynamic scheduling, and ergonomic assessment. This study demonstrates a proof-of-concept for introducing DH technology into DT-driven HRC for human-centered production systems.

Cybersecurity of Robotic Systems: Leading Challenges and Robotic System Design Methodology

Recent years have seen a rapid development of the Internet of Things (IoT) and the growth of autonomous robotic applications which are using network communications. Accordingly, an increasing advancement of intelligent devices with wireless sensors (that means autonomous robotic platforms) operating in challenging environments makes robots a tangible reality in the near future. Unfortunately, as a result of technical development, security problems emerge, especially when considering human–robot collaboration. Two abnormalities often compromise the basic security of collaborative robotic fleets: (a) Information faults and (b) system failures. This paper attempts to describe the methodology of a control framework design for secure robotic systems aided by the Internet of Things. The suggested concept represents a control system structure using blocks as the components. The structure is designed for the robots expected to interact with humans safely and act connected by communication channels. The properties of the components and relations between them are briefly described. The novelty of the proposed concept concerns the security mechanisms. The paper also categorizes two different modes of network attacks summarizing their causal effects on the human–robot collaboration systems. The issue of standardization is also raised. In particular, the works of the National Institute of Standards and Technology (NIST) and European Parliament (EP) on the security templates for communication channels are commented.

Development and validation of guidelines for safety in human-robot collaborative assembly systems

Industrial collaborative robotics is one of the most promising technologies of Industry 4.0. In particular, human-robot collaboration in assembly will be particularly interesting for manufacturing companies. In this context, the interaction between humans and robots opens new possibilities but also challenges. A major problem is related to safety: unwanted and unexpected contacts between the human and the robotic system may cause injuries and therefore limit the potential for collaboration. Nowadays, there is a lack of simple and practical tools for helping system designers in overcoming such limiting conditions. In this work, guidelines for the design of safe human-robot collaborative assembly are developed and classified, particularly focusing on the features characterizing the entire system. These are validated by means of a laboratory case study and a digital twin. The validation process is based on the assumption that a team of manufacturing engineers (not-experts in occupational health and safety) should be able to autonomously and gradually apply the given guidelines reducing the mechanical risk in a collaborative assembly system. The proposed solutions have been virtually modelled allowing the evaluation of their effectiveness. According to the results, the proposed guidelines effectively help non-expert users in the development and improvement of collaborative assembly systems from the safety perspective.

Status Recognition Using Pre-Trained YOLOv5 for Sustainable Human-Robot Collaboration (HRC) System in Mold Assembly

Molds are still assembled manually because of frequent demand changes and the requirement for comprehensive knowledge related to their high flexibility and adaptability in operation. We propose the application of human-robot collaboration (HRC) systems to improve manual mold assembly. In the existing HRC systems, humans control the execution of robot tasks, and this causes delays in the operation. Therefore, we propose a status recognition system to enable the early execution of robot tasks without human control during the HRC mold assembly operation. First, we decompose the mold assembly operation into task and sub-tasks, and define the actions representing the status of sub-tasks. Second, we develop status recognition based on parts, tools, and actions using a pre-trained YOLOv5 model, a one-stage object detection model. We compared four YOLOv5 models with and without a freezing backbone. The YOLOv5l model without a freezing backbone gave the optimal performance with a mean average precision (mAP) value of 84.8% and an inference time of 0.271 s. Given the success of the status recognition, we simulated the mold assembly operations in the HRC environment and reduced the assembly time by 7.84%. This study improves the sustainability of the mold assembly from the point of view of human safety, with reductions in human workload and assembly time.

Decision-Making Framework for Implementing Safer Human-Robot Collaboration Workstations: System Dynamics Modeling

Human-Robot Collaboration (HRC) systems are often implemented seeking for reducing risk of Work-related Musculoskeletal Disorders (WMSD) development and increasing productivity. The challenge is to successfully implement an industrial HRC to manage those factors, considering that non-linear behaviors of complex systems can produce counterintuitive effects. Therefore, the aim of this study was to design a decision-making framework considering the key ergonomic methods and using a computational model for simulations. It considered the main systemic influences when implementing a collaborative robot (cobot) into a production system and simulated scenarios of productivity and WMSD risk. In order to verify whether the computational model for simulating scenarios would be useful in the framework, a case study in a manual assembly workstation was conducted. The results show that both cycle time and WMSD risk depend on the Level of Collaboration (LoC). The proposed framework helps deciding which cobot to implement in a context of industrial assembly process. System dynamics were used to understand the actual behavior of all factors and to predict scenarios. Finally, the framework presented a clear roadmap for the future development of an industrial HRC system, drastically reducing risk management in decision-making.

Ergonomics and Human Factors as a Requirement to Implement Safer Collaborative Robotic Workstations: A Literature Review

There is a worldwide interest in implementing collaborative robots (Cobots) to reduce work-related Musculoskeletal Disorders (WMSD) risk. While prior work in this field has recognized the importance of considering Ergonomics & Human Factors (E&HF) in the design phase, most works tend to highlight workstations’ improvements due to Human-Robot Collaboration (HRC). Based on a literature review, the current study summarises studies where E&HF was considered a requirement rather than an output. In this article, the authors are interested in understanding the existing studies focused on Cobots’ implementation with ergonomic requirements, and the methods applied to design safer collaborative workstations. This review was performed in four prominent publications databases: Scopus, Web of Science, Pubmed, and Google Scholar, searching for the keywords ‘Collaborative robots’ or ‘Cobots’ or ‘HRC’ and ‘Ergonomics’ or ‘Human factors’. Based on the inclusion criterion, 20 articles were reviewed, and the main conclusions of each are provided. Additionally, the focus was given to the segmentation between studies considering E&HF during the design phase of HRC systems and studies applying E&HF in real-time on HRC systems. The results demonstrate the novelty of this topic, especially of the real-time applications of ergonomics as a requirement. Globally, the results of the reviewed studies showed the potential of E&HF requirements integrated into HRC systems as a relevant input for reducing WMSD risk.

Trust as Extended Control: Human-Machine Interactions as Active Inference.

In order to interact seamlessly with robots, users must infer the causes of a robot’s behavior–and be confident about that inference (and its predictions). Hence, trust is a necessary condition for human-robot collaboration (HRC). However, and despite its crucial role, it is still largely unknown how trust emerges, develops, and supports human relationship to technological systems. In the following paper we review the literature on trust, human-robot interaction, HRC, and human interaction at large. Early models of trust suggest that it is a trade-off between benevolence and competence; while studies of human to human interaction emphasize the role of shared behavior and mutual knowledge in the gradual building of trust. We go on to introduce a model of trust as an agent’ best explanation for reliable sensory exchange with an extended motor plant or partner. This model is based on the cognitive neuroscience of active inference and suggests that, in the context of HRC, trust can be casted in terms of virtual control over an artificial agent. Interactive feedback is a necessary condition to the extension of the trustor’s perception-action cycle. This model has important implications for understanding human-robot interaction and collaboration–as it allows the traditional determinants of human trust, such as the benevolence and competence attributed to the trustee, to be defined in terms of hierarchical active inference, while vulnerability can be described in terms of information exchange and empowerment. Furthermore, this model emphasizes the role of user feedback during HRC and suggests that boredom and surprise may be used in personalized interactions as markers for under and over-reliance on the system. The description of trust as a sense of virtual control offers a crucial step toward grounding human factors in cognitive neuroscience and improving the design of human-centered technology. Furthermore, we examine the role of shared behavior in the genesis of trust, especially in the context of dyadic collaboration, suggesting important consequences for the acceptability and design of human-robot collaborative systems.

A Supernumerary Soft Robotic Limb for Reducing Hand-Arm Vibration Syndromes Risks.

The most common causes of the risk of work-related musculoskeletal disorders (WMSD) have been identified as joint overloading, bad postures, and vibrations. In the last two decades, various solutions ranging from human-robot collaborative systems to robotic exoskeletons have been proposed to mitigate them. More recently, a new approach has been proposed with a high potential in this direction: the supernumerary robotic limbs SRLs are additional robotic body parts (e.g., fingers, legs, and arms) that can be worn by the workers, augmenting their natural ability and reducing the risks of injuries. These systems are generally proposed in the literature for their potentiality of augmenting the user’s ability, but here we would like to explore this kind of technology as a new generation of (personal) protective equipment. A supernumerary robotic upper limb, for example, allows for indirectly interacting with hazardous objects like chemical products or vibrating tools. In particular, in this work, we present a supernumerary robotic limbs system to reduce the vibration transmitted along the arms and minimize the load on the upper limb joints. For this purpose, an off-the-shelf wearable gravity compensation system is integrated with a soft robotic hand and a custom damping wrist, designed starting from theoretical considerations on a mass-spring-damper model. The real efficacy of the system was experimentally tested within a simulated industrial work environment, where seven subjects performed a drilling task on two different materials. Experimental analysis was conducted according to the ISO-5349. Results showed a reduction from 40 to 60% of vibration transmission with respect to the traditional hand drilling using the presented SRL system without compromising the time performance.

Human–Robot Collaboration Trends and Safety Aspects: A Systematic Review

Smart manufacturing and smart factories depend on automation and robotics, whereas human–robot collaboration (HRC) contributes to increasing the effectiveness and productivity of today’s and future factories. Industrial robots especially in HRC settings can be hazardous if safety is not addressed properly. In this review, we look at the collaboration levels of HRC and what safety actions have been used to address safety. One hundred and ninety-three articles were identified from which, after screening and eligibility stages, 46 articles were used for the extraction stage. Predefined parameters such as: devices, algorithms, collaboration level, safety action, and standards used for HRC were extracted. Despite close human and robot collaboration, 25% of all reviewed studies did not use any safety actions, and more than 50% did not use any standard to address safety issues. This review shows HRC trends and what kind of functionalities are lacking in today’s HRC systems. HRC systems can be a tremendously complex process; therefore, proper safety mechanisms must be addressed at an early stage of development.

Towards proactive human–robot collaboration: A foreseeable cognitive manufacturing paradigm

Human–robot collaboration (HRC) has attracted strong interests from researchers and engineers for improved operational flexibility and efficiency towards mass personalization. Nevertheless, existing HRC development mainly undertakes either human-centered or robot-centered manner reactively, where operations are conducted by following the pre-defined instructions, thus far from an efficient integration of robotic automation and human cognitions. The prevailing research on human-level information processing of cognitive computing, the industrial IoT, and robot learning creates the possibility of bridging the gap of knowledge distilling and information sharing between onsite operators, robots and other manufacturing systems. Hence, a foreseeable informatics-based cognitive manufacturing paradigm, Proactive HRC, is introduced as an advanced form of Symbiotic HRC with high-level cognitive teamwork skills to be achieved stepwise, including: (1) inter-collaboration cognition, establishing bi-directional empathy in the execution loop based on a holistic understanding of humans and robots’ situations; (2) spatio-temporal cooperation prediction, estimating human–robot–object interaction of hierarchical sub-tasks/activities over time for the proactive planning; and (3) self-organizing teamwork, converging knowledge of distributed HRC systems for self-organization learning and task allocation. Except for the description of their technical cores, the main challenges and potential opportunities are further discussed to enable the readiness towards Proactive HRC.

Formally-based Model-Driven Development of Collaborative Robotic Applications

The development of Human Robot Collaborative (HRC) systems faces many challenges. First, HRC systems should be adaptable and re-configurable to support fast production changes. However, in the development of HRC applications safety considerations are of paramount importance, as much as classical activities such as task programming and deployment. Hence, the reconfiguration and reprogramming of executing tasks might be necessary also to fulfill the desired safety requirements. Model-based software engineering is a suitable means for agile task programming and reconfiguration. We propose a model-based design-to-deployment toolchain that simplifies the routine of updating or modifying tasks. This toolchain relies on (i) UML profiles for quick model design, (ii) formal verification for exhaustive search for unsafe situations (caused by intended or unintended human behavior) within the model, and (iii) trans-coding tools for automating the development process. The toolchain has been evaluated on a few realistic case studies. In this paper, we show a couple of them to illustrate the applicability of the approach.