Human-robot Collaboration Research Articles

Prediction of cognitive conflict during unexpected robot behavior under different mental workload conditions in a physical human–robot collaboration

Objective. Brain–computer interface (BCI) technology is poised to play a prominent role in modern work environments, especially a collaborative environment where humans and machines work in close proximity, often with physical contact. In a physical human robot collaboration (pHRC), the robot performs complex motion sequences. Any unexpected robot behavior or faulty interaction might raise safety concerns. Error-related potentials, naturally generated by the brain when a human partner perceives an error, have been extensively employed in BCI as implicit human feedback to adapt robot behavior to facilitate a safe and intuitive interaction. However, the integration of BCI technology with error-related potential for robot control demands failure-free integration of highly uncertain electroencephalography (EEG) signals, particularly influenced by the physical and cognitive state of the user. As a higher workload on the user compromises their access to cognitive resources needed for error awareness, it is crucial to study how mental workload variations impact the error awareness as it might raise safety concerns in pHRC. In this study, we aim to study how cognitive workload affects the error awareness of a human user engaged in a pHRC. Approach. We designed a blasting task with an abrasive industrial robot and manipulated the mental workload with a secondary arithmetic task of varying difficulty. EEG data, perceived workload, task and physical performance were recorded from 24 participants moving the robot arm. The error condition was achieved by the unexpected stopping of the robot in 33% of trials. Main results. We observed a diminished amplitude for the prediction error negativity (PEN) and error positivity (Pe), indicating reduced error awareness with increasing mental workload. We further observed an increased frontal theta power and increasing trend in the central alpha and central beta power after the unexpected robot stopping compared to when the robot stopped correctly at the target. We also demonstrate that a popular convolution neural network model, EEGNet, could predict the amplitudes of PEN and Pe from the EEG data prior to the error. Significance. This prediction model could be instrumental in developing an online prediction model that could forewarn the system and operators of the diminished error awareness of the user, alluding to a potential safety breach in error-related potential-based BCI system for pHRC. Therefore, our work paves the way for embracing BCI technology in pHRC to optimally adapt the robot behavior for personalized user experience using real-time brain activity, enriching the quality of the interaction.

An experimental evaluation of robot-stopping approaches for improving fluency in collaborative robotics

AbstractThis paper explores and experimentally compares the effectiveness of robot-stopping approaches based on the speed and separation monitoring for improving fluency in collaborative robotics. In the compared approaches, a supervisory controller checks the distance between the bounding volumes enclosing human operator and robot and prevents potential collisions by determining the robot’s stop time and triggering a stop trajectory if necessary. The methods are tested on a Franka Emika robot with 7 degrees of freedom, involving 27 volunteer participants, who are asked to walk along assigned paths to cyclically intrude the robot workspace, while the manipulator is working. The experimental results show that scaling online the dynamic safety zones is beneficial for improving fluency of human-robot collaboration, showing significant statistical differences with respect to alternative approaches.

Imposing Motion Variability for Ergonomic Human-Robot Collaboration

OCCUPATIONAL APPLICATIONS “Overassistive” robots can adversely impact long-term human-robot collaboration in the workplace, leading to risks of worker complacency, reduced workforce skill sets, and diminished situational awareness. Ergonomics practitioners should thus be cautious about solely targeting widely adopted metrics for improving human-robot collaboration, such as user trust and comfort. By contrast, introducing variability and adaptation into a collaborative robot’s behavior could prove vital in preventing the negative consequences of overreliance and overtrust in an autonomous partner. This work reported here explored how instilling variability into physical human-robot collaboration can have a measurably positive effect on ergonomics in a repetitive task. A review of principles related to this notion of “stimulating” robot behavior is also provided to further inform ergonomics practitioners of existing human-robot collaboration frameworks.

Fog Cloud Computing and IoT Integration for AI enabled Autonomous Systems in Robotics

Fog Cloud Computing and the Internet of Things are transforming robotics by empowering AI-enabled autonomous systems. This study analyzes the benefits, drawbacks, and uses of this integration. AI-enabled autonomous robots can use edge computing and cloud resources for real-time data processing and decision-making, improving their performance and adaptability. Communication protocols, data management, security, and scalability are examined in the ecosystem. Case studies reveal how this confluence affects robotics applications. This research shows how FCC, IoT, and AI may improve robotic systems' efficiency, intelligence, and autonomy. The article covers AI-enabled autonomous systems in transportation, manufacturing, healthcare, agriculture, and smart cities. These technologies can improve productivity and safety in many fields, from self-driving automobiles to surgical robots. Integrating these technologies raises safety, ethical decision-making, data privacy, and security concerns. The report emphasizes transparent and ethical AI algorithms, unbiased decision-making, and regulatory frameworks to enable responsible integration and mitigate dangers. In the future, AI-enabled autonomous systems will be shaped by improved AI algorithms, multi-modal sensing, human-robot collaboration, and edge intelligence. It emphasizes the necessity of interdisciplinary collaboration and ethical considerations in responsible technology development. This study concludes with a detailed analysis of fog/cloud computing, IoT, and AI in robotics, revealing the immense promise and problems of AI-enabled autonomous systems. Responsible development and collaboration can help us negotiate this transformational frontier and create a safer, more efficient, and innovative society with AI-driven autonomous systems.

Dynamic Affect-Based Motion Planning of a Humanoid Robot for Human-Robot Collaborative Assembly in Manufacturing

The objective was to investigate the impacts of the robot’s dynamic affective expressions in task-related scenarios on human–robot collaboration (HRC) and performance in human–robot collaborative assembly tasks in flexible manufacturing. A human–robot hybrid cell was developed to facilitate a human co-worker and a robot to collaborate to assemble a few parts into a final product. The collaborative robot was a humanoid manufacturing robot with the ability to display its affective states due to changes in task scenarios on its face. The assembly task was divided into several subtasks, and based on an optimization strategy, the subtasks were optimally allocated to the human and the robot. A computational model of the robot’s affective states was derived inspired by that of humans following the biomimetic approach, and an affect-based motion planning strategy for the robot was proposed to enable the robot to adjust its motions and behaviors with task situations and communicate (inform) the situations to the human co-worker through affective expressions. The HRC and the assembly performance for the affect-based motion planning were experimentally evaluated based on a comprehensive evaluation scheme and were compared with two alternative conditions: (i) motion planning that did not display affective states, and (ii) motion planning that displayed text messages instead of displaying affective states to communicate the situations to the human co-worker. The results clearly showed that the dynamic affect-based motion planning produced significantly better HRC and assembly performance than that produced by motion planning associated with the display of no affective states or text messages. The results encouraged employing manufacturing robots with dynamic affective expressions to collaborate with humans in flexible assembly in manufacturing to improve HRC and assembly performance.

An Improved Moth-Flame Algorithm for Human–Robot Collaborative Parallel Disassembly Line Balancing Problem

In the context of sustainable development strategies, the recycling of discarded products has become increasingly important with the development of electronic technology. Choosing the human–robot collaborative disassembly mode is the key to optimizing the disassembly process and ensuring maximum efficiency and benefits. To solve the problem of human–robot cooperative parallel dismantling line balance, a mixed integer programming model is established and verified by CPLEX. An improved Moth-Flame Optimization (IMFO) algorithm is proposed to speed up convergence and optimize the disassembly process of various products. The effectiveness of IMFO is evaluated through multiple cases and compared with other heuristics. The results of these comparisons can provide insight into whether IMFO is the most appropriate algorithm for the problem presented.

PRogramAR: Augmented Reality End-User Robot Programming

The field of end-user robot programming seeks to develop methods that empower non-expert programmers to task and modify robot operations. In doing so, researchers may enhance robot flexibility and broaden the scope of robot deployments into the real world. We introduce PRogramAR (Programming Robots using Augmented Reality), a novel end-user robot programming system that combines the intuitive visual feedback of augmented reality (AR) with the simplistic and responsive paradigm of trigger-action programming (TAP) to facilitate human-robot collaboration. Through PRogramAR, users are able to rapidly author task rules and desired reactive robot behaviors, while specifying task constraints and observing program feedback contextualized directly in the real world. PRogramAR provides feedback by simulating the robot’s intended behavior and providing instant evaluation of TAP rule executability to help end users better understand and debug their programs during development. In a system validation, 17 end users ranging from ages 18 to 83 used PRogramAR to program a robot to assist them in completing three collaborative tasks. Our results demonstrate how merging the benefits of AR and TAP using elements from prior robot programming research into a single novel system can successfully enhance the robot programming process for non-expert users.

Human Understanding and Perception of Unanticipated Robot Action in the Context of Physical Interaction

Anticipating a future scenario where the robot initiates its own actions and behaves voluntarily when collaborating with humans, our research focuses on human understanding and perception of unanticipated robot actions during physical human-robot interaction. While the current literature searches for key factors that make the human-robot collaboration successful, the question of how people experience the robot’s unanticipated action as cooperative or uncooperative seems to remain open. We designed a game-based experiment (N = 35) where the participant played a “catch-falling-coins” game by moving a robotic arm. Our experiment introduced unanticipated robot actions in an “active session” where the robot targeted higher-valued coins without first informing the participants. Through semi-structured interviews and statistical analysis of questionnaires (Big Five Personality Test, SAM, NARS and CH33), we examined the participants’ understanding of the robot’s “intention” and their positive or negative perception of the robot as cooperative or uncooperative. Among the participants who understood that the robot’s “intention” was to catch the higher-valued coins, the majority of them reported a positive perception of the robot (cooperative or helpful) while this was not the case among those who did not understand the robot’s intention. We also observed relevant relationships between some personality traits and a person’s understanding of the robot’s intention. Qualitative analysis of the interviews allowed us to structure the process of perception change during the game into three phases: confusion, investigation, and adaptation. We believe that our research contributes to the study of human perception, and particularly to the relationship between a human’s understanding of unanticipated robot actions and their positive or negative perception of the robot.

Research on Human-Robot Collaboration Method for Parallel Robots Oriented to Segment Docking.

In the field of aerospace, large and heavy cabin segments present a significant challenge in assembling space engines. The substantial inertial force of cabin segments' mass often leads to unexpected motion during docking, resulting in segment collisions, making it challenging to ensure the accuracy and quality of engine segment docking. While traditional manual docking leverages workers' expertise, the intensity of the labor and low productivity are impractical for real-world applications. Human-robot collaboration can effectively integrate the advantages of humans and robots. Parallel robots, known for their high precision and load-bearing capacity, are extensively used in precision assembly under heavy load conditions. Therefore, human-parallel-robot collaboration is an excellent solution for such problems. In this paper, a framework is proposed that is easy to realize in production, using human-parallel-robot collaboration technology for cabin segment docking. A fractional-order variable damping admittance control and an inverse dynamics robust controller are proposed to enhance the robot's compliance, responsiveness, and trajectory tracking accuracy during collaborative assembly. This allows operators to dynamically adjust the robot's motion in real-time, counterbalancing inertial forces and preventing collisions between segments. Segment docking assembly experiments are performed using the Stewart platform in this study. The results show that the proposed method allows the robot to swiftly respond to interaction forces, maintaining compliance and stable motion accuracy even under unknown interaction forces.

Human-Robot handover task intention recognition framework by fusing human digital twin and deep domain adaptation

Industry 5.0 is conceived to achieve the expected high level of smart manufacturing and the more proactive and adaptive human-robot collaboration (HRC). In particular, human-robot handover tasks (HRHTs) are a notably crucial aspect of HRC, and resolving how to motivate robots to comprehend human handover intentions constitutes an exigent issue that demands resolution. To address these problems, A framework that integrates hierarchical human digital twin (HHDT) and deep domain adaptation is proposed, where HHDT model is built to digitise the human physical and physiological state and enhance the robot's perception of humans, a feature extractor-based on bidirectional long-short-term memory and spatio-temporal graph convolution network (BiLSTM-ST-GCN) is devised to extract latent spatio-temporal features of HRHTs intentions. Additionally, the framework incorporates a deep domain adaptation layer (DDAL) that facilitates knowledge transfer of HRHTs intentions by accounting for the distribution differences between the source and target operator objects. Extensive experiments demonstrate the superior performance of the proposed framework for HRHTs intention recognition and extraction of domain invariant features.

Balancing and scheduling of assembly line with multi-type collaborative robots

Human–robot collaboration (HRC) is a promising production mode that is in line with the vision of human-centered Industry 4.0. HRC contributes to the improvement of productivity as well as the reduction of workers’ ergonomic risk. In this study, we present one of the first attempts to address the assembly line balancing problem with multi-type collaborative robots (cobots), which allows human and robots to perform tasks in parallel or in collaboration. A mixed-integer programming (MIP) model is formulated to minimize the cycle time and a tight lower bound is proposed. We further propose a multi-objective model and an extended model to expand the scope of the study. Due to the complexity, an adaptive neighborhood simulated annealing algorithm (ANSA) is developed with the designed neighborhood operators and structures. Furthermore, an adaptive mechanism is applied to the ANSA to dynamically update the weights of the neighborhood structures based on historical information. Extensive computational experiments and a real case study are conducted to verify the superiority of ANSA. We further compare the application of diverse collaboration modes, i.e., sequential, simultaneous and supportive modes. The results also indicate that a suitable type of robot can improve productivity and the utilization rate of robots.

Sensorless admittance control of 6-DoF parallel robot in human-robot collaborative assembly

This paper presents an adaptive admittance control to achieve human-robot collaborative (HRC) assembling of large, heavy components without using external sensors. This sensorless adaptive admittance control is constructed by the dynamic model of a six degree-of-freedom (DoF) parallel robot based on finite and instantaneous screw (FIS) theory. The dynamic model is intuitive and easy to be programmed. After getting stable actuation force by Kalman filter and identifying friction by LuGre model, accurate force estimator is built. Admittance controller is adopted to convert the change of estimated force to the change in position, velocity and acceleration. As a result, the 6-DoF parallel robot is capable of following the guidance of operator. Particularly, adaptive admittance control is design by using robot velocity as reference. A fast motion before contacting and a fine motion during contacting are allowed. Experiments are conducted to verify force estimator, admittance control and adaptive admittance control for HRC assembly. Results show that external force is accurately estimated. Robot complaint control in response to operator's force is achieved. The adaptive admittance control enhances execution time, accuracy of the HRC assembly.

Collaborative robots (cobots) for disaster risk resilience: a framework for swarm of snake robots in delivering first aid in emergency situations.

Cobots are robots that are built for human-robot collaboration (HRC) in a shared environment. In the aftermath of disasters, cobots can cooperate with humans to mitigate risks and increase the possibility of rescuing people in distress. This study examines the resilient and dynamic synergy between a swarm of snake robots, first responders and people to be rescued. The possibility of delivering first aid to potential victims dispersed around a disaster environment is implemented. In the HRC simulation framework presented in this study, the first responder initially deploys a UAV, swarm of snake robots and emergency items. The UAV provides the first responder with the site planimetry, which includes the layout of the area, as well as the precise locations of the individuals in need of rescue and the aiding goods to be delivered. Each individual snake robot in the swarm is then assigned a victim. Subsequently an optimal path is determined by each snake robot using the A* algorithm, to approach and reach its respective target while avoiding obstacles. By using their prehensile capabilities, each snake robot adeptly grasps the aiding object to be dispatched. The snake robots successively arrive at the delivering location near the victim, following their optimal paths, and proceed to release the items. To demonstrate the potential of the framework, several case studies are outlined concerning the execution of operations that combine locomotion, obstacle avoidance, grasping and deploying. The Coppelia-Sim Robotic Simulator is utilised for this framework. The analysis of the motion of the snake robots on the path show highly accurate movement with and without the emergency item. This study is a step towards a holistic semi-autonomous search and rescue operation.

Augmented reality‐based method for road maintenance operators in human–robot collaborative interventions

AbstractRoad maintenance operators often work in dangerous environments and are in need of a support system to enhance their safety and efficiency. Augmented reality (AR) has proven to be useful in providing support to operators in various industrial sectors. However, the vast majority of the existing applications focus mainly on static, controlled environments, such as industrial shopfloors, although the dynamic flow of information that AR can provide could be very valuable to unstructured, dynamic environments. This paper presents a novel AR—based methodology for human–robot collaboration, real‐time instructions, and support for road maintenance operations, aiming to enhance operator safety and efficiency. A robot operating system‐based architecture is exploited for the communication of the modules. The methodology is tested in a laboratory environment, and the results validate the hypothesized enhancement of road operators. In the future, the application will be validated on real interventions in highways.

Model and metaheuristic for human–robot collaboration assembly line worker assignment and balancing problem

Given the Industry 5.0 trend towards human-centric and resilient industries, workers continue to be considered as a valuable and irreplaceable resource. Human–robot collaboration (HRC) is a promising production mode that combines the advantages of both human workers and robots, resulting in improved productivity and reduced ergonomic risks for workers. In this study, we present one of the first attempts to address the human–robot collaboration assembly line worker assignment and balancing problem (HRCALWABP), which allows workers and robots to perform tasks in parallel or in collaboration. Workers are heterogeneous, leading to variations in task execution times. We formulate a mixed-integer linear programming (MILP) model that minimizes the cycle time and propose a tight lower/upper bound. A rebalancing process is further considered in this study. Due to the complexity of HRCALWABP, we develop an improved adaptive neighborhood simulated annealing algorithm (IASA). An adaptive mechanism is applied to the IASA to dynamically adjust the selection probability of the designed neighborhood structures and operators, and a restart mechanism is developed for improving the search capacity. Extensive computational experiments are conducted to evaluate the performance of both the MILP and IASA, and the results confirm the superiority of the IASA compared to the MILP and other existing algorithms.

Human-Robot Collaboration Through a Multi-Scale Graph Convolution Neural Network With Temporal Attention

Human-Robot Collaboration Through a Multi-Scale Graph Convolution Neural Network With Temporal Attention

A Multimodality Scene Graph Generation Approach for Robust Human–Robot Collaborative Assembly Visual Relationship Representation

A Multimodality Scene Graph Generation Approach for Robust Human–Robot Collaborative Assembly Visual Relationship Representation

Human–Robot Collaboration Solution for Folding Fabrics Based on RGB-D Force and Kalman Filters

Human–Robot Collaboration Solution for Folding Fabrics Based on RGB-D Force and Kalman Filters

THE FUTURE IS NOW: AI POWERS NEXT-GENERATION ROBOTS

This article explores the transformative effect of synthetic intelligence (AI) on robotics, highlighting key regions where AI is revolutionizing the field. It discusses the evolution of robots, from simple automation to superior AI, and the position of AI in gaining knowledge of, sensing, reasoning, and communication for robots. The article also offers actual-global examples of AI robotics programs, along with self-riding automobiles, warehouse automation, and area exploration. It addresses the social implications of intelligent, self-sufficient robots, along with issues about process displacement, human relationships, and privacy. The article concludes by means of supplying interesting innovations on the horizon for AI robotics, such as smarter and more adaptable robots, more advantageous human-robot collaboration, miniaturization, self-reliant automobiles, and robotics for private use. Keywords: miniaturization, transformative, self-reliant automobiles.

Long Horizon Episodic Decision Making for Cognitively Inspired Robots

The Human decision-making process works by recollecting past sequences of observations and using them to decide the best possible action in the present. These past sequences of observations are stored in a derived form which only includes important information the brain thinks might be useful in the future, while forgetting the rest. we propose an architecture that tries to mimic the human brain and improve the memory efficiency of transformers by using a modified Transformer XL architecture which uses Automatic Chunking which only attends to the relevant chunks in the transformer block. On top of this, we use Forget Span which is technique to remove memories that do not contribute to learning. We also theorize the technique of Similarity based forgetting to remove repetitive memories. We test our model in various tasks that test the abilities required to perform well in a human-robot collaboration scenario.