Collaborative Robots Research Articles

A Survey of Wearable Lower Extremity Neurorehabilitation Exoskeleton: Sensing, Gait Dynamics, and Human–Robot Collaboration

A Survey of Wearable Lower Extremity Neurorehabilitation Exoskeleton: Sensing, Gait Dynamics, and Human–Robot Collaboration

Coordinated Transport by Dual Humanoid Robots Using Distributed Model Predictive Control.

Dual humanoid robot collaborative control systems possess better flexibility and adaptability in complex environments due to their similar structures to humans. This paper adopts a distributed model predictive controller based on the leader-follower approach to address the collaborative transportation control issue of dual humanoid robots. In the dual-robot collaborative control system, network latency issues may arise due to unstable network conditions, affecting the consistency of dual-robot collaboration. To solve this issue, a communication protocol was constructed through socket communication for dual-robot collaborative consistency, thereby resolving the problem of consistency in dual humanoid robot collaboration. Additionally, due to the complex structure of humanoid robots, there are deficiencies in position tracking accuracy during movement. To address the poor accuracy in position tracking, this paper proposes a distributed model predictive control that considers historical cumulative error, thus enhancing the position tracking accuracy of dual-robot collaborative control.

ARM Cortex Simulation Design for Trajectory Curves Evaluation of Collaborative Robots’ Tungsten Inert Gas Welding

ARM Cortex Simulation Design for Trajectory Curves Evaluation of Collaborative Robots’ Tungsten Inert Gas Welding

Balancing assembly lines with industrial and collaborative robots: Current trends and future research directions

Assembly-line balancing is a significant issue in production systems. Employing industrial robots as the main production resource was a milestone in developing assembly lines, and emerging Industry 4.0 led industries to build collaborative assembly lines by combining robots and human operator skills. Recently, the majority of research on assembly line balancing has contributed to addressing aspects of utilizing robots in assembly lines and how they can increase line performance. Various models and methods are developed, considering different objectives and performance indicators. Despite the increasing number of studies in this area, a thorough literature review is lacking in identifying gaps, shedding light on research directions, and facilitating future development. This study systematically reviews assembly-line balancing studies targeted at assembly lines with industrial and collaborative robots. Studies are classified based on their objectives and reviewed for their solution method, line layout, and other essential specifications. A descriptive analysis is provided to assist researchers and practitioners in linking different properties of assembly lines to the objectives and applied methodologies. The results show that most studies developed models and solution methods that focused on simultaneously optimizing more than one objective. The review reveals that minimizing the cycle time is the most popular objective, and meta-heuristic algorithms are the dominant solution approaches. It is also observed that balancing assembly lines with collaborative robots has received more attention in the last five years with the emergence of Industry 4.0. The review also highlights gaps in the related literature and provides promising insights for future research.

A vision-language-guided robotic action planning approach for ambiguity mitigation in human–robot collaborative manufacturing

Human–robot collaboration (HRC) has been recognized as a potent pathway towards mass personalization in the manufacturing sector, by leveraging the synergy of human creativity and robotic precision. Previous approaches rely heavily on visual perception to autonomously comprehend the HRC environment. However, the inherent ambiguity in human–robot communication cannot be consistently neutralized by relying solely on visual cues. With the recently soaring popularity of large language models (LLMs), the consideration of language data as a complementary information source has increasingly drawn research attention, while the application of such large models, particularly within the context of HRC in manufacturing, remains largely under-explored. In response to this gap, a vision-language reasoning approach is proposed to mitigate the communication ambiguity prevalent in human–robot collaborative manufacturing scenarios. A referred object retrieval model is first designed to alleviate the object–reference ambiguity in the human language command. This model is then seamlessly integrated into an LLM-based robotic action planner to achieve an improved HRC performance. The effectiveness of the proposed approach is demonstrated empirically through a series of experiments conducted on the object retrieval model and its application in a human–robot collaborative assembly case.

Assessing the Suitability of Automation Using the Methods–Time–Measurement Basic System

Due to its high complexity and the varied assembly processes, hybrid assembly systems characterized by human–robot collaboration (HRC) are meaningful. Suitable use cases must be identified efficiently to ensure cost-effectiveness and successful deployment in the respective assembly systems. This paper presents a method for evaluating the potential of HRC to derive automation suitability based on existing or to-be-collected time data. This should enable a quick and favorable statement to be made about processes, for efficient application in potential analyses. The method is based on the Methods–Time–Measurement Basic System (MTM-1) procedure, widely used in the industry. This ensures good adaptability in an industrial context. It extends existing models and examines how much assembly activities and processes can be optimized by efficiently allocating between humans and robots. In the process model, the assembly processes are subdivided and analyzed with the help of the specified MTM motion time system. The suitability of the individual activities and sub-processes for automation are evaluated based on criteria derived from existing methods. Two four-field matrices were used to interpret and classify the analysis results. The process is assessed using an example product from electrolyzer production, which is currently mainly assembled by hand. To achieve high statement reliability, further work is required to classify the results comprehensively.

Advancing human-robot collaboration: Predicting operator trajectories through AI and infrared imaging

In industrial applications demanding close interaction between humans and robots, accurate perception of the robot's surroundings is vital to adjust its behavior appropriately. This need becomes especially pertinent when manipulators operate autonomously and execute both separate and common tasks with human operators. To ensure operator safety in such Human-Robot Collaboration (HRC) environments, the present paper proposes an innovative approach that leverages advanced collaborative robotic systems capable of sensing the operator's position to effectively prevent or mitigate potential contact. Our solution combines infrared-thermal imaging technology with a workspace monitoring system, utilizing thermal and depth cameras, to track the operator's current location and predict their future trajectory using a multi-layer Long Short-Term Memory (LSTM) neural network. We also introduce a technique for simulating both human and robot movements to generate a synthetic dataset. This data is shared with the motion planning system to generate trajectories that do not intersect with the operator's future paths, minimizing downtime due to safety stoppages. The solution is integrated and evaluated within an industrial high payload collaborative robot context, and the results have shown improvements in cycle time efficiency and operator approval. This approach offers a promising direction for enhancing safety and productivity in HRC workspaces.

Autonomous control of an ultrasound probe for intra-operative ultrasonography using vision-based shape sensing of pneumatically attachable flexible rails

PurposeIn robotic-assisted minimally invasive surgery, surgeons often use intra-operative ultrasound to visualise endophytic structures and localise resection margins. This must be performed by a highly skilled surgeon. Automating this subtask may reduce the cognitive load for the surgeon and improve patient outcomes.MethodsWe demonstrate vision-based shape sensing of the pneumatically attachable flexible (PAF) rail by using colour-dependent image segmentation. The shape-sensing framework is evaluated on known curves ranging from r=30\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$r = 30$$\\end{document} to r=110\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$r = 110$$\\end{document} mm, replicating curvatures in a human kidney. The shape sensing is then used to inform path planning of a collaborative robot arm paired with an intra-operative ultrasound probe. We execute 15 autonomous ultrasound scans of a tumour-embedded kidney phantom and retrieve viable ultrasound images, as well as seven freehand ultrasound scans for comparison.ResultsThe vision-based sensor is shown to have comparable sensing accuracy with FBGS-based systems. We find the RMSE of the vision-based shape sensing of the PAF rail compared with ground truth to be 0.4975±0.4169\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$0.4975 \\pm 0.4169$$\\end{document} mm. The ultrasound images acquired by the robot and by the human were evaluated by two independent clinicians. The median score across all criteria for both readers was ‘3—good’ for human and ‘4—very good’ for robot.ConclusionWe have proposed a framework for autonomous intra-operative US scanning using vision-based shape sensing to inform path planning. Ultrasound images were evaluated by clinicians for sharpness of image, clarity of structures visible, and contrast of solid and fluid areas. Clinicians evaluated that robot-acquired images were superior to human-acquired images in all metrics. Future work will translate the framework to a da Vinci surgical robot.

Electro-mechanical modeling and identification of the UR5 e-series robot

Abstract Collaborative robotics is a field of growing industrial interest, within which understanding the energetic behavior of manipulators is essential. In this work, we present the electro-mechanical modeling of the UR5 e-series robot through the identification of its dynamics and electrical parameters. By means of the identified robot model, it is then possible to compute and optimize the energy consumption of the robot during prescribed trajectories. The proposed model is derived from data acquired from the robot controller during bespoke experimental tests, using model identification procedures and datasheet provided by manipulator, motors, and gearbox manufacturers. The entire procedure does not require the use of any additional sensor, so it can be easily replicated with an off-the-shelf manipulator, and applied to other robots of the same family.

Research Status and Development Trend of Multi-arm Collaborative Robots

Abstract: Industrial robots are mainly used in metal forming, automotive, and electrical and electronics industries. After decades of unremitting efforts, industrial robots have achieved great success. However, problems such as low safety, weak environmental adaptability, and insufficient humancomputer interaction ability of industrial robots have greatly limited the application of industrial robots, and multi-arm collaborative robots came into being in this case. Multi-arm collaborative robots offer high flexibility and precision. Over the years, the application and development of multiarm collaborative robots have been paid more and more attention. This paper aims to summarize the patents related to multi-arm cooperative robots, including their principles, classification, latest progress and future development, in order to meet the needs of the real world. According to the structural characteristics of multi-arm collaborative robots and the requirements of multi-arm collaborative robots application field, the most typical mobile collaborative robots and stationary dual-arm collaborative robots in multi-arm collaborative robots are summarized. By investigating various patents of multi-arm collaborative robots the main problems of multi-arm collaborative robots are summarized and analyzed, such as collisions or conflicts between robotic arms, low power-to-weight ratio, loose structures, and heavy weight of the robot body. In addition, the development trend of multi-arm collaborative robots is also discussed. Multi-arm robots have broad prospects and potential in future applications and development. With the advancement of new technologies, multi-arm collaborative robots will become more efficient, intelligent, safe, and flexible. More relevant patents are expected to be invented in the future.

Assembly complexity and physiological response in human-robot collaboration: Insights from a preliminary experimental analysis

Industry 5.0 paradigm has renewed interest in the human sphere, emphasizing the importance of workers’ well-being in manufacturing activities. In such context, collaborative robotics originated as a technology to support humans in tiring and repetitive tasks. This study investigates the effects of assembly complexity in Human-Robot collaboration using physiological indicators of cognitive effort. In a series of experiments, participants performed assembly processes of different products with varying complexity, in two modalities: manually and with cobot assistance. Physiological measures, including skin conductance, heart rate variability and eye-tracking metrics were collected. The analysis of physiological signals showed trends suggesting the impact of assembly complexity and cobot support. One key finding of the study is that a single physiological signal usually may not provide a complete understanding of cognitive load. Therefore, a holistic approach should be followed. This approach highlighted the importance of considering multiple measures simultaneously to accurately assess workers’ well-being in industrial environments.

Real-time collision detection based on external torque mutation suppression and time series analysis

PurposeAs the demand for human–robot collaboration in manufacturing applications grows, the necessity for collision detection functions in robots becomes increasingly paramount for safety. Hence, this paper aims to improve the existing method to achieve efficient, accurate and sensitive robot collision detection.Design/methodology/approachThe external torque is estimated by momentum observers based on the robot dynamics model. Because the state of the joints is more accessible to distinguish under the action of the suppression operator proposed in this paper, the mutated external torque caused by joint reversal can be accurately attenuated. Finally, time series analysis (TSA) methods can continuously generate dynamic thresholds based on external torques.FindingsCompared with the collision detection method based only on TSA, the invalid time of the proposed method is less during joint reversal. Although the soft-collision detection accuracy of this method is lower than that of the symmetric threshold method, it is superior in terms of detection delay and has a higher hard-collision detection accuracy.Originality/valueOwing to the mutated external torque caused by joint reversal, which seriously affects the stability of time series models, the collision detection method based only on TSA cannot detect continuously. The consequences are disastrous if the robot collides with people or the environment during joint reversal. After multiple experimental verifications, the proposed method still exhibits detection capabilities during joint reversal and can implement real-time collision detection. Therefore, it is suitable for various engineering applications.

Data-efficient multimodal human action recognition for proactive human–robot collaborative assembly: A cross-domain few-shot learning approach

With the recent vision of Industry 5.0, the cognitive capability of robots plays a crucial role in advancing proactive human–robot collaborative assembly. As a basis of the mutual empathy, the understanding of a human operator’s intention has been primarily studied through the technique of human action recognition. Existing deep learning-based methods demonstrate remarkable efficacy in handling information-rich data such as physiological measurements and videos, where the latter category represents a more natural perception input. However, deploying these methods in new unseen assembly scenarios requires first collecting abundant case-specific data. This leads to significant manual effort and poor flexibility. To deal with the issue, this paper proposes a novel cross-domain few-shot learning method for data-efficient multimodal human action recognition. A hierarchical data fusion mechanism is designed to jointly leverage the skeletons, RGB images and depth maps with complementary information. Then a temporal CrossTransformer is developed to enable the action recognition with very limited amount of data. Lightweight domain adapters are integrated to further improve the generalization with fast finetuning. Extensive experiments on a real car engine assembly case show the superior performance of proposed method over state-of-the-art regarding both accuracy and finetuning efficiency. Real-time demonstrations and ablation study further indicate the potential of early recognition, which is beneficial for the robot procedures generation in practical applications. In summary, this paper contributes to the rarely explored realm of data-efficient human action recognition for proactive human–robot collaboration.

Forming a view: a human factors case study of augmented reality collaboration in assembly

Industry 4.0 technology is promoted as improving manufacturing flexibility, and competitiveness; though Australia has been slow to adopt. The Australian Navy shipbuilding program provides opportunities for accelerating technology adoption, revitalising manufacturing productivity and competitiveness. Adopting a sociotechnical systems lens, our research sought to identify usability, workload, and user experience of an augmented reality head-mounted display (AR-HMD) deployed to complete multiple work tasks in a workflow (electrical assembly, collaborative robot (cobot) mediated inspection, and remote troubleshooting using video call). Usability was rated ‘average’ (System Usability Scale mean = 69.8) and workload ‘acceptable’ (NASA Task Load Index mean = 25.8) for the AR-HMD alone, with usability of the integrated work system (IWS) rated ‘good’ (SUS mean = 79.2). Results suggest software interfaces, tracking, and gesturing methods for the AR-HMD require improvement. This trial shows the AR-HMD provides a versatile platform for integrating multiple digital technologies without hindering effectiveness of end-user performance, potentially benefiting productivity and quality.

A framework for human–robot collaboration enhanced by preference learning and ergonomics

Industry 5.0 aims to prioritize human operators, focusing on their well-being and capabilities, while promoting collaboration between humans and robots to enhance efficiency and productivity. The integration of collaborative robots must ensure the health and well-being of human operators. Indeed, this paper addresses the need for a human-centered framework proposing a preference-based optimization algorithm in a human–robot collaboration (HRC) scenario with an ergonomics assessment to improve working conditions. The HRC application consists of optimizing a collaborative robot end-effector pose during an object-handling task. The following approach (AmPL-RULA) utilizes an Active multi-Preference Learning (AmPL) algorithm, a preference-based optimization method, where the user is requested to iteratively provide qualitative feedback by expressing pairwise preferences between a couple of candidates. To address physical well-being, an ergonomic performance index, Rapid Upper Limb Assessment (RULA), is combined with the user’s pairwise preferences, so that the optimal setting can be computed. Experimental tests have been conducted to validate the method, involving collaborative assembly during the object handling performed by the robot. Results illustrate that the proposed method can improve the physical workload of the operator while easing the collaborative task.

A Voice-Enabled ROS2 Framework for Human–Robot Collaborative Inspection

Quality inspection plays a vital role in current manufacturing practice since the need for reliable and customized products is high on the agenda of most industries. Under this scope, solutions enhancing human–robot collaboration such as voice-based interaction are at the forefront of efforts by modern industries towards embracing the latest digitalization trends. Current inspection activities are often based on the manual expertise of operators, which has been proven to be time-consuming. This paper presents a voice-enabled ROS2 framework towards enhancing the collaboration of robots and operators under quality inspection activities. A robust ROS2-based architecture is adopted towards supporting the orchestration of the process execution flow. Furthermore, a speech recognition application and a quality inspection solution are deployed and integrated to the overall system, showcasing its effectiveness under a case study deriving from the automotive industry. The benefits of this voice-enabled ROS2 framework are discussed and proposed as an alternative way of inspecting parts under human–robot collaborative environments. To measure the added value of the framework, a multi-round testing process took place with different parameters for the framework’s modules, showcasing reduced cycle time for quality inspection processes, robust HRI using voice-based techniques and accurate inspection.

Evolution of Hybrid Cellular Automata for Density Classification Problem

This paper describes a solution for the image density classification problem (DCP) using an entirely distributed system with only local processing of information named cellular automata (CA). The proposed solution uses two cellular automata’s features, density conserving and translation of the information stored in the cellular automata’s cells through the lattice, in order to obtain the solution for the density classification problem. The motivation for choosing a bio-inspired technique based on CA for solving the DCP is to investigate the principles of self-organizing decentralized computation and to assess the capabilities of CA to achieve such computation, which is applicable to many real-world decentralized problems that require a decision to be taken by majority voting, such as multi-agent holonic systems, collaborative robots, drones’ fleet, image analysis, traffic optimization, forming and then separating clusters with different values. The entire application is coded using the C# programming language, and the obtained results and comparisons between different cellular automata configurations are also discussed in this research.

Air–ground cooperative multi-target searching under an unknown urban environment

The collaboration of heterogeneous multiple robots has been shown to be capable of significantly enhancing the system redundancy, autonomy, robustness, and so on. However, realizing the collaboration with specific tasks in practice often requires the development of sophisticated mechanisms which are envisioned to exploit distinct benefits of the heterogeneous platforms. Thus, we propose a novel air–ground cooperative framework in this paper for the task of multi-target searching under an unknown urban environment. In particular, a group of unmanned aerial vehicles (UAVs) is employed to operate above the urban area to provide surveillance from a global perspective. Under the guidance of UAVs, multiple teams of unmanned ground vehicles (UGVs) are deployed to conduct the target searching missions. The UAVs’ and UGVs’ searching strategies are devised correspondingly leveraging on their own advantageous features. Finally, an ingenious integration of UAVs’ and UGVs’ searching operations is established by a notion of the upper confidence bound. Simulation results are provided to demonstrate the effectiveness of our approach.

Prediction of Human Reaching Pose Sequences in Human–Robot Collaboration

Abstract In human–robot collaboration, robots and humans must work together in shared, overlapping, workspaces to accomplish tasks. If human and robot motion can be coordinated, then collisions between robot and human can seamlessly be avoided without requiring either of them to stop work. A key part of this coordination is anticipating humans’ future motion so robot motion can be adapted proactively. In this work, a generative neural network predicts a multi-step sequence of human poses for tabletop reaching motions. The multi-step sequence is mapped to a time-series based on a human speed versus motion distance model. The input to the network is the human’s reaching target relative to current pelvis location combined with current human pose. A dataset was generated of human motions to reach various positions on or above the table in front of the human starting from a wide variety of initial human poses. After training the network, experiments showed that the predicted sequences generated by this method matched the actual recordings of human motion within an L2 joint error of 7.6 cm and L2 link roll–pitch–yaw error of 0.301 rad on average. This method predicts motion for an entire reach motion without suffering from the exponential propagation of prediction error that limits the horizon of prior works.

Vision-based adaptive LT sliding mode admittance control for collaborative robots with actuator saturation

Abstract In this paper, we propose a novel vision-based adaptive leakage-type (LT) sliding mode admittance control for actuator-constrained collaborative robots to realize the synchronous control of the precise path following and compliant interaction force. Firstly, we develop a vision-admittance-based model to couple the visual feedback and force sensing in the image feature space so that a reference image feature trajectory can be obtained concerning the contact force command and predefined trajectory. Secondly, considering the system uncertainty, external disturbance, and torque constraints of collaborative robots in reality, we propose an adaptive sliding mode controller in the image feature space to perform precise trajectory tracking. This controller employs a leakage-type (LT) adaptive control law to reduce the side effects of system uncertainties without knowing the upper bound of system uncertainties. Moreover, an auxiliary dynamic is considered in this controller to overcome the joint torque constraints. Finally, we prove the convergence of the tracking error with the Lyapunov stability analysis and operate various semi-physical simulations compared to the conventional adaptive sliding mode and parallel vision/force controller to demonstrate the efficacy of the proposed controller. The simulation results show that compared with the controller mentioned above, the path following accuracy and interaction force control precision of the proposed controller increased by 50% and achieved faster convergence.