Collaborative Robots Research Articles

Designing interaction interface for supportive human-robot collaboration: A co-creation study involving factory employees

Considering social-technical factors during the design and implementation of collaborative robots (cobots) is important to ensure their successful integration into industrial workspaces and the well-being of the operators such as ergonomics. In this work, we present a co-creation study in developing an interaction interface for a human–robot collaboration (HRC) system involving SME factory employees in the Netherlands. Employing a qualitative research method, the co-creation activities in this study sought employees’ input on preferred use cases and collaboration methods with robots. The gathered qualitative data was used to design the HRC interaction interface, aligning it with operators’ needs and preferences. The developed system was fully functional, underwent technical validation, and received feedback from the factory operators. Our study emphasizes the importance of involving employees in the design of HRC interaction interfaces, which can result in HRC systems that meet their needs and preferences. Such customized systems have the potential to enhance the acceptance of robots in industrial settings. The study contributes to the field by demonstrating a participatory approach to designing an HRC interaction interface for a robot in a real industrial setting, where no use cases or interaction modalities were pre-defined.

Nighttime Harvesting of OrBot (Orchard RoBot)

The Robotics Vision Lab of Northwest Nazarene University has developed the Orchard Robot (OrBot), which was designed for harvesting fruits. OrBot is composed of a machine vision system to locate fruits on the tree, a robotic manipulator to approach the target fruit, and a gripper to remove the target fruit. Field trials conducted at commercial orchards for apples and peaches during the harvesting season of 2021 yielded a harvesting success rate of about 85% and had an average harvesting cycle time of 12 s. Building upon this success, the goal of this study is to evaluate the performance of OrBot during nighttime harvesting. The idea is to have OrBot harvest at night, and then human pickers continue the harvesting operation during the day. This human and robot collaboration will leverage the labor shortage issue with a relatively slower robot working at night. The specific objectives are to determine the artificial lighting parameters suitable for nighttime harvesting and to evaluate the harvesting viability of OrBot during the night. LED lighting was selected as the source for artificial illumination with a color temperature of 5600 K and 10% intensity. This combination resulted in images with the lowest noise. OrBot was tested in a commercial orchard using twenty Pink Lady apple trees. Results showed an increased success rate during the night, with OrBot gaining 94% compared to 88% during the daytime operations.

Human-cobot collaboration's impact on success, time completion, errors, workload, gestures and acceptability during an assembly task

The 5.0 industry promotes collaborative robots (cobots). This research studies the impacts of cobot collaboration using an experimental setup. 120 participants realized a simple and a complex assembly task. 50% collaborated with another human (H/H) and 50% with a cobot (H/C). The workload and the acceptability of the cobotic collaboration were measured. Working with a cobot decreases the effect of the task complexity on the human workload and on the output quality. However, it increases the time completion and the number of gestures (while decreasing their frequency). The H/C couples have a higher chance of success but they take more time and more gestures to realize the task. The results of this research could help developers and stakeholders to understand the impacts of implementing a cobot in production chains.

Quicker Path planning of a collaborative dual-arm robot using Modified BP-RRT* algorithm

Path-planning of an industrial robot is an important task to reduce the overall operation time. In industrial tasks, path planning is executed with lead-through programming, where in most cases the robot faces singulated object configurations. Cluttered environments demand path-planning algorithms, which are sensor driven, rather than pre-programmed. Path-planning algorithms, like RRT, and RRT* and their variants have inherent problems like the duration of a search and the creation of several node samples which may lead to longer path lengths. Back Propagation-Rapidly exploring Random Tree* (BP-RRT*) algorithm was a leap forward when an obstacle is enveloped with a sphere. Due to the spherical envelope of the obstacle, this method evaluates the connection between the path and obstacle in space with a spherical envelope using the triangular function and identifies the non-collision path in 3D space. This predicts the best non-collision path in the 3D workspace. The current state-of-the-art of BP-RRT* is limited to single-arm robots. A collaborative dual-arm robot faces more problems in path planning than a single-arm robot like inter-collision of manipulator arms apart from avoiding obstacles. A Modified BP-RRT* algorithm is proposed for the dual-arm collaborative robot has a pre-stage partition of grids that makes the computation faster, efficient, and collision-free compared to the traditional path planning algorithms namely RRT, RRT*, Improved RRT* and BP-RRT*. The algorithm is implemented in simulation as well as in physical implementation for ABB YuMi dual-arm collaborative robot and the typical length of the path of the proposed modified BP-RRT* method has reduced by 53.8% from the traditional RRT method, 6.95% from the RRT* method, 7.77% from improved RRT* method and 6.83% from the BP-RRT* method. Also, the average time to grasp has reduced by 17.84%, the typical duration for search has decreased by 33.45%, the number of node samples created has reduced by 14.79% from BP-RRT* algorithm.

Towards an ontology to capture human attributes in human-robot collaboration

AbstractA core predicate of Industry 5.0 (I5.0) is the integration of human, environmental and social factors with new technologies. The integration of collaborative robots offers increased productivity but raises questions on safety and how robots can respond to varying cognitive and physical attributes. This paper discusses the significance of structured ontologies in managing complex information for proactive, safe, and productive human-robot collaboration. The paper highlights the future work to be undertaken to ensure the safe and fluid integration of humans and robots within I5.0.

Assessing Trust in Construction AI-Powered Collaborative Robots Using Structural Equation Modeling

Assessing Trust in Construction AI-Powered Collaborative Robots Using Structural Equation Modeling

A matrix-based approach to step-wise assess the safety of collaborative robots in manufacturing

AbstractCollaborative robots (cobots) allow for flexible manufacturing, supporting more customised product designs. Although safety is key for socio-technical human-cobot workplaces, existing safety assessment support like standards and guidelines require extensive experience and can be experienced as overwhelming. To make cobot risk assessments more accessible, especially for novices, and increase traceability from hazard to risk to mitigation, this paper presents a matrix-based approach that decomposes this daunting activity into smaller better manageable steps.

An LLM-based vision and language cobot navigation approach for Human-centric Smart Manufacturing

Industry 5.0 prioritizes Human-centric Smart Manufacturing (HSM), aiming to enhance human operators’ well-being and needs. This necessitates collaborative robots with advanced natural interaction capabilities and improved perception, cognition, and action intelligence. The Large Language Model (LLM) exhibits strong reasoning abilities and generalization capabilities, which can significantly advance the development of HSM once integrated into human–robot interaction and collaboration. Accordingly, this paper explores part of LLM’s ability in the context of smart manufacturing, focusing on addressing interruptions in the manufacturing process caused by repetitive tool fetching. To alleviate this issue, a vision and language cobot navigation approach is innovatively adopted in the manufacturing environment that could be further used to assist operators in retrieving tools. Specifically, a real Human–Robot Collaboration (HRC) manufacturing scene is first reconstructed and annotated using Three-Dimensional (3D) point cloud techniques. Then the LLM is utilized for the Automated Guided Vehicle (AGV) to comprehend natural language commands and generate Python code to triger navigation actions. Finally, the Pathfinder algorithm is applied for corresponding path planning. The framework is implemented in the Artificial Intelligence (AI) Habitat simulator, and the case studies demonstrate that the AGV can accurately comprehend complex language instructions, empowering human operators to complete manufacturing tasks efficiently.

Order Picking Optimization in Smart Warehouses With Human–Robot Collaboration

Order Picking Optimization in Smart Warehouses With Human–Robot Collaboration

LaserDex: Improvising Spatial Tasks Using Deictic Gestures and Laser Pointing for Human–Robot Collaboration in Construction

LaserDex: Improvising Spatial Tasks Using Deictic Gestures and Laser Pointing for Human–Robot Collaboration in Construction

Enhancing human–robot collaborative transportation through obstacle-aware vibrotactile warning and virtual fixtures

Transporting large and heavy objects can benefit from Human–Robot Collaboration (HRC), increasing the contribution of robots to our daily tasks and addressing challenges arising from labor shortages. This strategy typically positions the human collaborator as the leader, with the robot assuming the follower role. However, when transporting large objects, the operator’s situational awareness can be compromised as the objects may occlude different parts of the environment, weakening the human leader’s decision-making capacity and leading to failure due to collision. This paper proposes a situational awareness framework for collaborative transportation to face this challenge. The framework integrates a multi-modal haptic-based Obstacle Feedback Module with two units. The first unit consists of a warning module that alerts the operator through a haptic belt with four vibrotactile devices that provide feedback about the location and proximity of the obstacles. The second unit implements virtual fixtures as hard constraints for mobility. The warning feedback and the virtual fixtures act online based on the information given by two Lidars mounted on a mobile manipulator to detect the obstacles in the surroundings. By enhancing the operator’s awareness of the environment, the proposed module improves the safety of the human–robot team in collaborative transportation scenarios by preventing collisions. Experiments with 16 non-expert subjects in four feedback modalities during four scenarios report an objective evaluation thanks to quantitative metrics and subjective evaluations based on user-level experiences. The results reveal the strengths and weaknesses of the implemented feedback modalities while providing solid evidence of the increased situational awareness of the operator when the two haptic units are employed.

Enhancing awareness of industrial robots in collaborative manufacturing

The diffusion of Human-Robot Collaborative cells is prevented by several barriers. Classical control approaches seem not yet fully suitable for facing the variability conveyed by the presence of human operators beside robots. The capabilities of representing heterogeneous knowledge representation and performing abstract reasoning are crucial to enhance the flexibility of control solutions. To this aim, the ontology SOHO (Sharework Ontology for Human-Robot Collaboration) has been specifically designed for representing Human-Robot Collaboration scenarios, following a context-based approach. This work brings several contributions. This paper proposes an extension of SOHO to better characterize behavioral constraints of collaborative tasks. Furthermore, this work shows a knowledge extraction procedure designed to automatize the synthesis of Artificial Intelligence plan-based controllers for realizing flexible coordination of human and robot behaviors in collaborative tasks. The generality of the ontological model and the developed representation capabilities as well as the validity of the synthesized planning domains are evaluated on a number of realistic industrial scenarios where collaborative robots are actually deployed.

Managing safety of the human on the factory floor: a computer vision fusion approach

PurposeMaintaining the safety of the human is a major concern in factories where humans co-exist with robots and other physical tools. Typically, the area around the robots is monitored using lasers. However, lasers cannot distinguish between human and non-human objects in the robot’s path. Stopping or slowing down the robot when non-human objects approach is unproductive. This research contribution addresses that inefficiency by showing how computer-vision techniques can be used instead of lasers which improve up-time of the robot.Design/methodology/approachA computer-vision safety system is presented. Image segmentation, 3D point clouds, face recognition, hand gesture recognition, speed and trajectory tracking and a digital twin are used. Using speed and separation, the robot’s speed is controlled based on the nearest location of humans accurate to their body shape. The computer-vision safety system is compared to a traditional laser measure. The system is evaluated in a controlled test, and in the field.FindingsComputer-vision and lasers are shown to be equivalent by a measure of relationship and measure of agreement. R2 is given as 0.999983. The two methods are systematically producing similar results, as the bias is close to zero, at 0.060 mm. Using Bland–Altman analysis, 95% of the differences lie within the limits of maximum acceptable differences.Originality/valueIn this paper an original model for future computer-vision safety systems is described which is equivalent to existing laser systems, identifies and adapts to particular humans and reduces the need to slow and stop systems thereby improving efficiency. The implication is that computer-vision can be used to substitute lasers and permit adaptive robotic control in human–robot collaboration systems.

Supporting decision-making of collaborative robot (cobot) adoption: The development of a framework

Collaborative robots (cobots) are emerging in manufacturing as a response to the current mass customization production paradigm and the fifth industrial revolution. Before adopting this technology in production processes and benefiting from its advantages, manufacturers need to analyze the investment. Therefore, this study aims to develop a decision-making framework for cobot adoption, incorporating a comprehensive set of quantitative and qualitative criteria, to be used by decision-makers in manufacturing companies. To achieve that objective, a qualitative study was conducted by collecting data through interviews with key actors in the cobot (or advanced manufacturing technologies) adoption decision process in manufacturing companies. The main findings of this study include, firstly, an extensive list of decision criteria, as well as some indicators to be used by decision-makers, some of which are new to the literature. Secondly, a decision-making framework for cobot adoption is proposed, as well as a set of guidelines to use it. The framework is based on a weighted scoring method and can be customizable by the manufacturing company depending on its specific context, needs, and resources. The main contribution of this study consists in assisting decision-makers of manufacturing companies in performing more complete and sustained decision analyses regarding cobots adoption.

OmniCharger: CNN-based Hand Gesture Interface to Operate an Electric Car Charging Robot through Teleconference

The automation of the car charging process is motivated by the rapid development of technologies for self-driving cars and the increasing importance of ecological transportation units. Automation of this process requires the implementation of Computer Vision (CV) techniques. However, it remains challenging to precisely position the charger plug autonomously due to the sensitivity of CV algorithms to lighting and weather conditions. We introduce a novel robotic operation system based on hand gesture recognition through teleconferencing software. The users, connected by teleconference, use their hand gestures to teleoperate the electric plug located on the collaborative robot end-effector. We conducted a user study to evaluate the system performance and suitability using OmniCharger and two baseline interfaces (a UR10 Teach Pendant and a Logitech F710 Wireless Gamepad). Except for two trials, all the users were able to locate the plug inside of a 5 cm target using the interfaces. The distance to the target and the orientation error did not present statistically significant differences ( \(p=0.1099\,{\gt}\,0.05\) and \(p=0.0903\,{\gt}\,0.05\) , respectively) in the use of the three interfaces. The NASA TLX questionnaire results showed low values in all the sub-classes, the SUS results rated the usability of the proposed interface above average (68%), and the UEQ showed excellent performance of the OmniCharger interface in the attractiveness, stimulation, and novelty attributes.

Advancing human–robot collaboration in handcrafted manufacturing: cobot-assisted polishing design boosted by virtual reality and human-in-the-loop

Industry 5.0 envisions a future where seamless collaboration between humans and robots enhances efficiency, innovation, and coevolution. While collaborative robots have found widespread applications in manufacturing, particularly in tasks like pick-and-place and assembly, their integration into handcrafted manufacturing processes presents unique challenges. This article focuses on advancing technology in the less-explored field of cobot-assisted handcrafted manufacturing, specifically in the fashion industry, with a priority on reducing work-related risks. In handcrafted processes, which often involve intricate and artistic work, cobots face challenges related to nuanced decision-making, adaptability to customizations, and the need for precise manual dexterity. The study delves into the cobot-assisted polishing of leather shoes, addressing issues associated with product delicacy, process and knowledge formalization, versatility, and integration into existing manufacturing processes. To overcome these challenges, the research proposes the application of cobots in the initial polishing phase, which is the most physically demanding, allowing artisans to focus on finalization, quality control, and process supervision. The study also applies the concept of human-in-the-loop (HITL) and virtual reality simulation to optimize collaboration, ensuring safety, ergonomics, and efficiency. The article contributes to the scientific and industrial communities by pioneering the study of collaborative robotics in craftsmanship, successfully implementing human–robot collaboration (HRC) in an industrial setting, demonstrating the effectiveness of virtual simulation and HITL, and prioritizing human factors throughout the design and development of HRC. The insights gained from this research are crucial for achieving practical solutions in industrial environments while aligning with the performance objectives of companies and workers’ well-being.

A Soft Collaborative Robot for Contact-based Intuitive Human Drag Teaching.

Soft material-based robots, known for their safety and compliance, are expected to play an irreplaceable role in human-robot collaboration. However, this expectation is far from real industrial applications due to their complex programmability and poor motion precision, brought by the super elasticity and large hysteresis of soft materials. Here, a soft collaborative robot (Soft Co-bot) with intuitive and easy programming by contact-based drag teaching, and also with exceptional motion repeatability (< 0.30% of body length) and ultra-low hysteresis (< 2.0%) is reported. Such an unprecedented capability is achieved by a biomimetic antagonistic design within a pneumatic soft robot, in which cables are threaded to servo motors through tension sensors to form a self-sensing system, thus providing both precise actuation and dragging-aware collaboration. Hence, the Soft Co-bots can be first taught by human drag and then precisely repeat various tasks on their own, such as electronics assembling, machine tool installation, etc. The proposed Soft Co-bots exhibit a high potential for safe and intuitive human-robot collaboration in unstructured environments, promoting the immediate practical application of soft robots.

Ultracompact Joint Torque Sensor Units With Enhanced Resolution for Collaborative Robots

Ultracompact Joint Torque Sensor Units With Enhanced Resolution for Collaborative Robots

Designing Digital Twins of Robots Using Simscape Multibody

Digital twins of industrial and collaborative robots are widely used to evaluate and predict the behavior of manipulators under different control strategies. However, these digital twins often employ simplified mathematical models that do not fully describe their dynamics. In this paper, we present the design of a high-fidelity digital twin of a six degrees-of-freedom articulated robot using Simscape Multibody, a Matlab toolbox that allows the design of robotic manipulators in a rather intuitive and user-friendly manner. This robot digital twin includes joint friction, transmission gears, and electric actuators dynamics. After assessing the dynamic accuracy of the Simscape model, we used it to test a computed torque control scheme, proving that this model can be reliably used in simulations with different aims, such as validating control schemes, evaluating collaborative functions or minimizing power consumption.

Unleashing mixed-reality capability in Deep Reinforcement Learning-based robot motion generation towards safe human–robot collaboration

The integration of human–robot collaboration yields substantial benefits, particularly in terms of enhancing flexibility and efficiency within a range of mass-personalized manufacturing tasks, for example, small-batch customized product inspection and assembly/disassembly. Meanwhile, as human–robot collaboration lands broader in manufacturing, the unstructured scene and operator uncertainties are increasingly involved and considered. Consequently, it becomes imperative for robots to execute in a safe and adaptive manner rather than solely relying on pre-programmed instructions. To tackle it, a systematic solution for safe robot motion generation in human–robot collaborative activities is proposed, leveraging mixed-reality technologies and Deep Reinforcement Learning. This solution covers the entire process of collaboration starting with an intuitive interface that facilitates bare-hand task command transmission and scene coordinate transformation before the collaboration begins. In particular, mixed-reality devices are employed as effective tools for representing the state of humans, robots, and scenes. This enables the learning of an end-to-end Deep Reinforcement Learning policy that addresses both the uncertainties in robot perception and decision-making in an integrated manner. The proposed solution also implements policy simulation-to-reality deployment, along with motion preview and collision detection mechanisms, to ensure safe robot motion execution. It is hoped that this work could inspire further research in human–robot collaboration to unleash and exploit the powerful capabilities of mixed reality.