Collaborative Robots Research Articles

Dynamic Bandwidth Allocation for Collaborative Multi-Robot Systems Based on Task Execution Measures

Multi-robot systems (MRSs) is a growing field of research that focuses on the collaboration of multiple robots to achieve a common global objective. Managing these systems poses several challenges, including coordination, task allocation, and communication. Among these challenges, a major area of focus is devising an effective communication scheme that ensures robots’ cooperation and adapts to varying conditions during task execution. In this paper, we develop a novel communication management framework tailored for MRSs, specifically addressing dynamic bandwidth distribution in networked teleoperated robotic systems. The algorithm is combined with semi-autonomous formation control based on the Artificial Potential Fields (APF) algorithm, which allows each individual robot to avoid local obstacles autonomously and tries to maintain a desired formation with its neighbors, while the operator is in charge of high-level control only. Common Dynamic Bandwidth Allocation (DBA) algorithms allocate bandwidth to different units based on network conditions and requirements. On the other hand, our proposed DBA scheme dynamically distributes the available bandwidth on communication streams based on factors related to task execution and system performance. In specific, bandwidth is allocated in a way that adapts to changes occurring in the system’s environment and its internal state, including the effect of the autonomous action taken by the path planner on the MRS and the performance of the controller of each individual robot. By addressing the limitations of existing approaches through shaping the communication behavior of the MRS based on performance measures, our proposed algorithm offers a promising solution for improving the performance and efficiency of MRSs. The proposed scheme is tested through simulations on a group of six unmanned aerial vehicles (UAVs) in the Robot Operating System (ROS)-Gazebo simulation environment. The obtained results show the scheme’s capability for enhancing the robotic system’s performance while significantly reducing bandwidth consumption. Experimental testing on two mobile robots further demonstrates the effectiveness of the proposed scheme.

Impacts of Collaborative Robots on Construction Work Performance and Worker Perception: Experimental Analysis of Human–Robot Collaborative Wood Assembly

Impacts of Collaborative Robots on Construction Work Performance and Worker Perception: Experimental Analysis of Human–Robot Collaborative Wood Assembly

Effects of the Human Presence among Robots in the ARIAC 2023 Industrial Automation Competition

ARIAC is a robotic simulation competition promoted by NIST annually since 2017, aiming to present competitors’ with contemporary industry problems to be solved using agile robotics. For the 2023 competition, ARIAC competitors must perform assembly and kitting tasks by controlling four autonomous ground vehicles (AGVs), one floor-based robot, and one ceiling-based (Gantry) robot in an attempt to overcome a range of agility challenges in the supplied simulated environment, itself based on the Robot Operating System (ROS 2) and Gazebo. The 2023 competition also included a “human” agility challenge, comprising a (simulated) human operator working among robots on the factory floor. This development was motivated by the fact that, while robots and automation play an increasingly significant role in modern manufacturing, there still remains a close relationship between machines and humans. They should complement each other’s strengths and cover each other’s limitations while also observing any required safety rules. For example, the ISO standard “Robots and Robotic Devices – Collaborative robots” (ISO 15066:2016) prescribes the distances required between humans and robots. Within the ARIAC simulation environment, each human operator is controlled using autonomous Belief-Desire-Intention (BDI) agents. At the same time, competitors can monitor the position of each human operator at any time by subscribing to the relevant ROS topic. In this article, we analyse the effects of this (simulated) human presence in the 2023 ARIAC competition and perform a detailed analysis of how the three different human personalities that were implemented affect the assembly tasks undertaken at the four different locations of the assembly stations. Given how the system is currently implemented, it appears that the influence of each encoded personality on the competitors is not as predictable as anticipated. We expand on why this may be a problem when addressing real collaborative spaces involving humans and industrial robots and the improvements that can be undertaken to mitigate the ensuing problems.

An adaptive genetic algorithm for the optimization of multi-mobile robot collaboration

The proposed approach utilizes the Robot Operating System (ROS) to simulate multi-robot collaboration across various scenarios, ensuring rigorous testing and validation of the algorithm. Our simulation environment encompasses complex tasks such as 3D digitalization, which demand precise coordination and efficient resource management among robots. The adaptive genetic algorithm (GA) continuously adjusts its parameters to improve performance, making it highly suitable for dynamic and unpredictable environments. Our results demonstrate that the adaptive GA significantly enhances the efficiency and effectiveness of Multi-Robot Task Allocation (MRTA) compared to traditional methods that lack optimization. By incorporating a cost function with various weighted factors, the task allocation process becomes both comprehensive and adaptable to specific mission requirements. This ensures that the robots can allocate tasks efficiently, even as conditions change. This study underscores the potential of adaptive genetic algorithms to advance the capabilities of mobile multi-robot systems, particularly in applications that require high levels of collaboration and precision. Our approach not only improves task allocation efficiency but also enhances the overall coordination and performance of the robotic system. The adaptability and robustness of the GA make it a promising solution for real-world applications, including search-and-rescue missions, environmental monitoring, and industrial automation. So, the potential of adaptive genetic algorithm presents a significant advancement in optimizing mobile multi-robot collaboration. Also, its ability to dynamically adjust to changing conditions and improve task allocation processes highlights its potential for a wide range of applications, marking a notable step forward in the field of collaborative robotics.

On flange-based 3D hand–eye calibration for soft robotic tactile welding

This paper investigates the direct application of standardized designs on the robot for conducting robot hand–eye calibration by employing 3D scanners with collaborative robots. The well-established geometric features of the robot flange are exploited by directly capturing its point cloud data. In particular, an iterative method is proposed to facilitate point cloud processing towards a refined calibration outcome. Several extensive experiments are conducted over a range of collaborative robots, including Universal Robots UR5 & UR10 e-series, Franka Emika, and AUBO i5 using an industrial-grade 3D scanner Photoneo Phoxi S & M and a commercial-grade 3D scanner Microsoft Azure Kinect DK. Experimental results show that translational and rotational errors converge efficiently to less than 0.28 mm and 0.25 degrees, respectively, achieving a hand–eye calibration accuracy as high as the camera’s resolution, probing the hardware limit. A welding seam tracking system is presented, combining the flange-based calibration method with soft tactile sensing. The experiment results show that the system enables the robot to adjust its motion in real-time, ensuring consistent weld quality and paving the way for more efficient and adaptable manufacturing processes.

Open-source design of low-cost sensorised elastic actuators for collaborative prosthetics and orthotics

Collaborative robots, or cobots, have become popular due to their ability to safely operate alongside humans in shared environments. These robots use compliant actuators as a key design element to prevent damage during unintended collisions. In prosthetic and orthotic applications, compliant actuators are crucial for ensuring user safety and comfort. However, most compliant cobots for these applications are excessively expensive and complex to construct. Our study introduces an innovative, cost-effective, and sensorised elastic actuator design tailored for prosthetics and orthotics. The design uses a modular approach and leverages 3D printing technology for rapid customisation, enabling efficient and affordable fabrication. Both hardware and software components are open-source, facilitating unrestricted access for students, researchers, and practitioners. Our design supports impedance and admittance control techniques, enhancing the system’s capabilities. Validation results show a standard deviation of 9.67 Nm between calculated and measured torque in impedance control and 0.2563 radians between calculated and measured angles in admittance control. This allows for improved adaptability to varying operational requirements in prosthetics and orthotics. By introducing this educational framework encompassing a low-cost, sensorised elastic actuator design, we aim to address the need for accessible solutions in the field of collaborative robotics for prosthetics and orthotics.

Dynamic scenario-enhanced diverse human motion prediction network for proactive human–robot collaboration in customized assembly tasks

Dynamic scenario-enhanced diverse human motion prediction network for proactive human–robot collaboration in customized assembly tasks

Neural admittance control based on motion intention estimation and force feedforward compensation for human–robot collaboration

Neural admittance control based on motion intention estimation and force feedforward compensation for human–robot collaboration

Evolution of Industrial Robots from the Perspective of the Metaverse: Integration of Virtual and Physical Realities and Human–Robot Collaboration

During the transition from Industry 4.0 to Industry 5.0, industrial robotics technology faces the need for intelligent and highly integrated development. Metaverse technology creates immersive and interactive virtual environments, allowing technicians to perform simulations and experiments in the virtual world, and overcoming the limitations of traditional industrial operations. This paper explores the application and evolution of metaverse technology in the field of industrial robotics, focusing on the realization of virtual–real integration and human–machine collaboration. It proposes a design framework for a virtual–real interaction system based on the ROS and WEB technologies, supporting robot connectivity, posture display, coordinate axis conversion, and cross-platform multi-robot loading. This paper emphasizes the study of two key technologies for the system: virtual–real model communication and virtual–real model transformation. A general communication mechanism is designed and implemented based on the ROS, using the ROS topic subscription to achieve connection and real-time data communication between physical robots and virtual models, and utilizing URDF model transformation technology for model invocation and display. Compared with traditional simulation software, i.e., KUKA Sim PRO (version 1.1) and RobotStudio (version 6.08), the system improves model loading by 45.58% and 24.72%, and the drive response by 41.50% and 28.75%. This system not only supports virtual simulation and training but also enables the operation of physical industrial robots, provides persistent data storage, and supports action reproduction and offline data analysis and decision making.

Enhancing Robot Behavior with EEG, Reinforcement Learning and Beyond: A Review of Techniques in Collaborative Robotics

Collaborative robotics is a major topic in current robotics research, posing new challenges, especially in human–robot interaction. The main aspect in this area of research focuses on understanding the behavior of robots when engaging with humans, where reinforcement learning is a key discipline that allows us to explore sophisticated emerging reactions. This review aims to delve into the relevance of different sensors and techniques, with special attention to EEG (electroencephalography data on brain activity) and its influence on the behavior of robots interacting with humans. In addition, mechanisms available to mitigate potential risks during the experimentation process such as virtual reality are also be addressed. In the final part of the paper, future lines of research combining the areas of collaborative robotics, reinforcement learning, virtual reality, and human factors are explored, as this last aspect is vital to ensuring safe and effective human–robot interactions.

Interactive Path Editing and Simulation System for Motion Planning and Control of a Collaborative Robot

Robots in hazardous environments demand precise and advanced motion control, making extensive simulations crucial for verifying the safety of motion planning. This paper presents a simulation system that enables interactive path editing, allowing for motion planning in a simulated collaborative robot environment and its real-world application. The system includes a simulation host, a control board, and a robot. Unity 3D on a Windows platform provides the simulation environment, while a virtual Linux environment runs ROS2 for execution. Unity sends edited motion paths to ROS2 using the Unity ROS TCP Connector package. The ROS2 MoveIt framework generates trajectories, which are synchronized back to Unity for simulation and real-world validation. To control the six-axis Indy7 collaborative robot, we used the MIO5272 embedded board as an EtherCAT master. Verified trajectories are sent to the target board, synchronizing the robot with the simulation in position and speed. Data are relayed from the host to the MIO5272 using ROS2 and the Data Distribution Service (DDS) to control the robot via EtherCAT communication. The system enables direct simulation and control of various trajectories for robots in hazardous environments. It represents a major advancement by providing safe and optimized trajectories through efficient motion planning and repeated simulations, offering a clear improvement over traditional time-consuming and error-prone teach pendant methods.

PyIgH : A unified architecture of IgH EtherCAT Master based on Python considering hard real-time constraints

The increasing demand for rapid application development tools, especially those employing high-level languages such as Python, has underscored the importance of utilizing a wide array of popular libraries while addressing real-time constraints in distributed hardware systems. This paper introduces PyIgH, a unified architecture of an IgH EtherCAT master based on Python, specifically designed to satisfy hard real-time requirements in an EtherCAT network. Implemented as a Python module, PyIgH exposes the functionalities and capabilities of an open-source EtherCAT master, facilitating seamless configuration and control of EtherCAT slave devices within the Python runtime environment. Real-time adaptation of the POSIX library, encapsulated within Python, is also utilized to satisfy the timing requirements of EtherCAT. The feasibility of the proposed approach is verified by analyzing the real-time performance in terms of periodicity and in-controller delay of the EtherCAT control task with a 1 kHz cycle. Experimental results demonstrate that PyIgH is suitable for hard real-time applications and serves as a valid alternative to conventional low-level EtherCAT masters. Additionally, a practical application involving motion control of a six-axis collaborative robot showcases consistent performance of PyIgH within a real-time multi-tasking environment.

Human–Robot Collaborative Manufacturing Cell with Learning-Based Interaction Abilities

This paper presents a collaborative manufacturing cell implemented in a laboratory setting, focusing on developing learning-based interaction abilities to enhance versatility and ease of use. The key components of the system include 3D real-time volumetric monitoring for safety, visual recognition of hand gestures for human-to-robot communication, classification of physical-contact-based interaction primitives during handover operations, and detection of hand–object interactions to anticipate human intentions. Due to the nature and complexity of perception, deep-learning-based techniques were used to enhance robustness and adaptability. The main components are integrated in a system containing multiple functionalities, coordinated through a dedicated state machine. This ensures appropriate actions and reactions based on events, enabling the execution of specific modules to complete a given multi-step task. An ROS-based architecture supports the software infrastructure among sensor interfacing, data processing, and robot and gripper controllers nodes. The result is demonstrated by a functional use case that involves multiple tasks and behaviors, paving the way for the deployment of more advanced collaborative cells in manufacturing contexts.

Experimental implementation of skeleton tracking for collision avoidance in collaborative robotics

Collaborative robotic manipulators can stop in case of a collision, according to the ISO/TS 15066 and ISO 10218-1 standards. However, in a human-robot collaboration scenario where the robot and the human share the workspace, a better solution with concerns to production and operator safety would be to perceive in advance the presence of an obstacle and be able to avoid it, thereby completing the task without halting the robot. In this paper, an obstacle avoidance algorithm is tested using a sensor system for real-time human detection; the operator represents a potential dynamic obstacle that can interfere with the robot motion. The sensor system consists of three RGB-D cameras. A custom software framework has been developed in Python exploiting machine learning tools for human skeleton detection. The coordinates of the human body joints relative to the manipulator base are used as input to the obstacle avoidance algorithm. The use of multiple sensors makes it possible to limit the occlusion problem; in addition, the choice of non-wearable sensors goes in the direction of better operator comfort. A series of experimental tests were performed to verify the accuracy of skeleton detection and the ability of the system to avoid obstacles in real time. The human motion caption system, in particular, was validated through a comparison with a commercial system based on wearable IMU sensors widely used and validated in motion capture. There is an NRMSE of 3.23%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$3.23\\%$$\\end{document} for the RGB-D camera-based skeleton detection system, against an NRMSE of 12.23%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$12.23\\%$$\\end{document} for the IMU wearable sensor system. Test results confirm that the system is able to avoid collisions with the human body under various conditions, static or dynamic, ensuring a minimum safety distance to any part of the manipulator. In 44 tests in which the operator moves around the robot with possible collisions (at a speed typical of manufacturing operations), the minimum operator-robot distance averaged 208mm\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$208\\,\ extrm{mm}$$\\end{document}, being 200mm\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$200\\,\ extrm{mm}$$\\end{document} the limit safety distance set by algorithm.

Possibilities of Automating the Additive Manufacturing Process of Material Extrusion – MEX

The article presents the possibilities of automating production pre-processing and post-processing operations for the Material Extrusion - MEX process based on Fused Filament Fabrication technology. Automation is based on hardware and software solutions. For this purpose, a special research station was developed, equipped with a warehouse of working platforms, a 3D printer and a collaborative robot that integrates individual elements of the manufacturing process. The developed solution allows for increasing the efficiency of the manufacturing cell and reducing the operator's involvement in manual operations at the pre-processing and post-processing stages.

SHOWMe: Robust object-agnostic hand-object 3D reconstruction from RGB video

In this paper, we tackle the problem of detailed hand-object 3D reconstruction from monocular video with unknown objects, for applications where the required accuracy and level of detail is important, e.g. object hand-over in human–robot collaboration, or manipulation and contact point analysis. While the recent literature on this topic is promising, the accuracy and generalization abilities of existing methods are still lacking. This is due to several limitations, such as the assumption of known object class or model for a small number of instances, or over-reliance on off-the-shelf keypoint and structure-from-motion methods for object-relative viewpoint estimation, prone to complete failure with previously unobserved, poorly textured objects or hand-object occlusions. To address previous method shortcomings, we present a 2-stage pipeline superseding state-of-the-art (SotA) performance on several metrics. First, we robustly retrieve viewpoints relying on a learned pairwise camera pose estimator trainable with a low data regime, followed by a globalized Shonan pose averaging. Second, we simultaneously estimate detailed 3D hand-object shapes and refine camera poses using a differential renderer-based optimizer. To better assess the out-of-distribution abilities of existing methods, and to showcase our methodological contributions, we introduce the new SHOWMe benchmark dataset with 96 sequences annotated with poses, millimetric textured 3D shape scans, and parametric hand models, introducing new object and hand diversity. Remarkably, we show that our method is able to reconstruct 100% of these sequences as opposed to SotA Structure-from-Motion (SfM) or hand-keypoint-based pipelines, and obtains reconstructions of equivalent or better precision when existing methods do succeed in providing a result. We hope these contributions lead to further research under harder input assumptions. The dataset can be downloaded at https://download.europe.naverlabs.com/showme.

Force and time-optimal trajectory planning for dual-arm unilateral cooperative grasping

This paper studies the dual-arm manipulation of an object by means of two collaborative robots. The latter hold the object through limited contact areas, thus applying unilateral contact constraints. This manipulation strategy increases versatility, since it does not require specific grippers depending on the object shape and size. However, to ensure grasping stability (i.e. no slipping of the object), a suitable internal force must be prescribed to ensure the fulfillment of the static-friction condition. In this work, the trend of the internal force is included among the inputs of a time-optimal trajectory planning, in order to find the minimal internal prestress that is able to both satisfy the static-friction condition and manipulate the object in minimal time. Admittance control is used to modulate the forces exerted by the robot end-effectors on the object. An extensive experimentation, on different 6-dimensional trajectories reaching linear and angular accelerations up to 4.5 m/s2 and 7.4 rad/s2, is presented and discussed.

Effect of human emotional responses on human-robot team pty in construction

Human-robot collaboration (HRC) is expected to improve construction productivity by integrating robotic capabilities with human expertise. However, the effects of HRC on human emotions are unclear despite their critical impacts on human performance during HRC. This paper examined the inter-relationship between robots' parameters, like movement speed, and emotional responses, during HRC for a construction task. Eighteen participants were observed as they experience various conditions of robots' parameters during a bricklaying task with an arm-type collaborative robot in a laboratory setting. Their brainwaves (i.e., electroencephalogram (EEG) signals) were monitored using an EEG headset to identify emotional responses. Results showed significant emotional variations due to robots' parameters, which could affect the estimated productivity of human-robot teams. This paper demonstrated the emotional changes caused by the robots' parameters during HRC for construction tasks, highlighting the importance of considering emotions as a way to balance the emotional well-being of people and the high productivity of human-robot teams.

A Framework for Enhanced Human–Robot Collaboration during Disassembly Using Digital Twin and Virtual Reality

A Framework for Enhanced Human–Robot Collaboration during Disassembly Using Digital Twin and Virtual Reality

Manipulation of a Complex Object Using Dual-Arm Robot with Mask R-CNN and Grasping Strategy

Hot forging is one of the common manufacturing processes for producing brass workpieces. However forging produces flash which is a thin metal part around the desired part formed with an excessive material. Using robots with vision system to manipulate this workpiece has encountered several challenging issues, e.g. the uncertain shape of flash, color, reflection of brass surface, different lighting condition, and the uncertainty surrounding the position and orientation of the workpiece. In this research, Mask region-based convolutional neural network together with image processing is used to resolve these issues. The depth camera can provide images for visual detection. Machine learning Mask region-based convolutional neural network model was trained with color images and the position of the object is determined by the depth image. A dual arm 7 degree of freedom collaborative robot with proposed grasping strategy is used to grasp the workpiece that can be in inappropriate position and pose. Eventually, experiments were conducted to assess the visual detection process and the grasp planning of the robot.