Robot Operating Research Articles

Using skeletal position to estimate human error rates in telemanipulator operators.

In current telerobotics and telemanipulator applications, operators must perform a wide variety of tasks, often with a high risk associated with failure. A system designed to generate data-based behavioural estimations using observed operator features could be used to reduce risks in industrial teleoperation. This paper describes a non-invasive bio-mechanical feature capture method for teleoperators used to trial novel human-error rate estimators which, in future work, are intended to improve operational safety by providing behavioural and postural feedback to the operator. Operator monitoring studies were conducted in situ using the MASCOT teleoperation system at UKAEA RACE; the operators were given controlled tasks to complete during observation. Building upon existing works for vehicle-driver intention estimation and robotic surgery operator analysis, we used 3D point-cloud data capture using a commercially available depth camera to estimate an operator's skeletal pose. A total of 14 operators were observed and recorded for a total of approximately 8 h, each completing a baseline task and a task designed to induce detectable but safe collisions. Skeletal pose was estimated, collision statistics were recorded, and questionnaire-based psychological assessments were made, providing a database of qualitative and quantitative data. We then trialled data-driven analysis by using statistical and machine learning regression techniques (SVR) to estimate collision rates. We further perform and present an input variable sensitivity analysis for our selected features.

Experimental Research on Avoidance Obstacle Control for Mobile Robots Using Q-Learning (QL) and Deep Q-Learning (DQL) Algorithms in Dynamic Environments

This study provides simulation and experimental results on techniques for avoiding static and dynamic obstacles using a deep Q-learning (DQL) reinforcement learning algorithm for a two-wheel mobile robot with independent control. This method integrates the Q-learning (QL) algorithm with a neural network, where the neural networks in the DQL algorithm act as approximators for the Q matrix table for each pair (state–action). The effectiveness of the proposed solution was confirmed through simulations, programming, and practical experimentation. A comparison was drawn between the DQL algorithm and the QL algorithm. Initially, the mobile robot was connected to the control script using the Robot Operating System (ROS). The mobile robot was programmed in Python within the ROS operating system, and the DQL controller was programmed in Gazebo software. The mobile robot underwent testing in a workshop with various experimental scenarios considered. The DQL controller displayed improvements in computation time, convergence time, trajectory planning accuracy, and obstacle avoidance. As a result, the DQL controller surpassed the QL algorithm in terms of performance.

An Indoor Localization System for Automotive Driving Competitions

Localization, in both indoor and outdoor settings, represents a problem that has received increased attention lately. This paper presents the development, testing, and validation of an indoor localization system for 1/10 scale vehicles based on the Robot Operating System (ROS) and ArUco marker detection. It has a distributed architecture, consisting of tens of Raspberry Pi (RPi) singleboard computers running ROS nodes and fitted with cameras, that monitor a certain area of the 14x14 m plane. The developed system has been successfully used in three editions of Bosch Future Mobility Challenge, an international student competition, where the participants are required to implement autonomous driving functionalities in an environment resembling a real-life city on small-scale automated vehicles.

Development of a dry mock-up system for verifying pyroprocess automation

This paper presents the design and operation of an autonomous robot for pyroprocess automation, which requires unique approaches beyond those used in industrial applications to achieve the desired performance. Maintaining an extremely dry atmosphere is crucial to handle various materials, including chloride, and an autonomous system ensures this dry environment. The drying room dehumidifier was carefully selected and designed to generate dry air, and different types of dry air conditioning performance were evaluated, including assessing worker accessibility inside the mock-up to determine the system's feasibility. Containers used for process materials were modified to fit the gripper system of the gantry robot for automation. The loading and unloading of process materials in each equipment were automatically performed to connect the process equipment with the robotic system. The gantry robot primarily operated through macro motion to approach waypoints containing process materials, eliminating the need for precise approach motion. The robot's tapered jaw design allowed it to grasp the target object even with imperfect positioning. Robot motions were programmed using a robot simulator for initial positioning and motion planning, and real accuracy was tested in a mock-up facility using the OPC platform.

Disturbance Observer-Based Model Predictive Control for an Unmanned Underwater Vehicle

This work addresses the motion control problem for a 4-degree-of-freedom unmanned underwater vehicle (UUV) in the presence of nonlinear dynamics, parametric uncertainties, system constraints, and time-varying external disturbances. A disturbance observer-based control scheme is proposed, which is structured around the model predictive control (MPC) method integrated with an extended active observer (EAOB). Compared to the conventional disturbance observer, the developed EAOB has the ability to handle both external disturbances and system/measurement noises simultaneously. The EAOB leverages a combination of sensor measurements and a system dynamic model to estimate disturbances in real-time, which allows continuous estimation and compensation of time-varying disturbances back to the controller. The proposed disturbance observer-based MPC is implemented by feeding the estimated disturbances back into the MPC’s prediction model, which forms an effective adaptive controller with a parameter-varying model. The proposed control strategy is validated through simulations in a Gazebo and robot operating system environment. The results show that the proposed method can effectively reject unpredictable disturbances and improve the UUV’s control performance.

Digitization of Accounting in the Innovative Management of Autonomous Robotic Transport

The digitization of economic processes is advancing across all sectors, contributing to the development of Industry 5.0. A key element of this fifth industrial revolution is the activation of robotic economic activity. Recently, advancements in autonomous robotic transport have been implemented in practice. However, both the practical application of unmanned vehicles and scientific developments in this field have shown low efficiency in the implementation of projects for the autonomous transportation of goods and passengers. This inefficiency stems from insufficient attention to the accounting and management aspects of autonomous robotic transport operations. The scientific and practical novelty of this study lies in improving accounting and management practices in the context of digitalization, specifically by addressing the fundamental transformations in economic processes caused by the use of autonomous vehicles. The key organizational factors influencing accounting for robotic transport operations include the type of transported objects, fuel and energy resource consumption, human involvement, the capacity and number of goods (or passengers) transported at one time, continuous operation, maintainability, software update capabilities, autonomous interaction with other transport means, and communication and information sharing with customers of transport services. A method for digitizing the accounting of fuel and energy costs, personnel wages, social activity deductions, depreciation, operational costs, and other costs related to the functioning of autonomous robotic transport has been developed. This method leverages IoT data and considers the organizational prerequisites mentioned. The use of two-dimensional calculation units, such as "kilogramkilometre" and "passenger-kilometre" units, for the digitalization of cost calculations for passenger and cargo transportation via autonomous robotic transport has been proposed. Additionally, the procedure for determining the cost of transport services for end users and the formation of information arrays for the innovative management of transport enterprises has been refined. The elimination of organizational restrictions in managing autonomous transport operations, alongside the need for information synchronization between transport enterprises and other business entities within the information ecosystem of a smart city, highlights future research prospects in this area.

Exploration of Civics Teaching in the ‘Industrial Robot Programming and Operation’ Course

In order to actualize moral and collaborative education, the reform of ideological and political aspects within professional courses becomes imperative. This paper explicates the necessity of instigating ideological and political education due to the singular nature of traditional teaching in the ‘Industrial Robot Programming and Operation’ course and its lack of emphasis on ideological and political elements. It extensively explores the embedding of ideological and political components within the course. By amplifying teachers’ awareness of ideological and political principles, refining the course’s training objectives, diversifying teaching methodologies, and enhancing the course evaluation mechanism, we implement specific practices that integrate ideology and politics into the course. The aim is to cultivate highly proficient technical and skilled individuals.

Kinematic Analysis of Multi-Section Opposite Magnetic Catheter Robots With Solution Multiplicity

Magnetic catheter robots have great potential for application in minimally invasive surgical interventions that require submillimeter soft catheters to be inserted into the body. Compared to other medical continuum robots, the kinematic characteristics of the magnetic continuum robot are required to be improved, including operating dexterity. The opposite magnetic continuum robot (OMCR), which has oppositely-magnetized magnets on the robot, is capable of deforming in large and high-order curvatures and shows improved dexterity. However, unstable kinematic behaviors emerge owing to solution multiplicity and affect the actual operation. For example, snapping, which indicates the sudden jump from one state to another, induces danger for medical operations. In addition, the motion sequence of the external mobile magnet affects the final state of OMCR. Therefore, in this study, we first analyze the reason for these emerging behaviors by understanding the solution multiplicity of an OMCR. We then modified the OMCR kinematic model with instability analysis and additional constraints, to predict the critical point for snapping and avoid other unexpected behavior for path planning. Herein, our study aims to analyze the stability of the OMCR with relatively unstable configurations in their actuation space to ensure stable and dexterous motions for medical applications. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The motivation of this work is to analyze, predict, and avoid the unexpected behaviors of the medical magnetic catheter robots, and ensure stable and dexterous operation. Submillimeter magnetic catheter robots have received considerable attention in the past ten years owing to their small size and dexterous motion. They have great potential for minimally invasive surgical interventions that require the navigation of “tighter corners” inside the human body. However, there may exist multiple local solutions under an external field, especially when the robot deforms in high-order curvature for dexterous operation. This situation induces unexpected behaviors and danger for medical use, such as a sudden jump from one state to another. Herein, we analyze the solution evolution as the external field varies and predict the unexpected behaviors using a modified kinematic model with instability analysis. Preliminary experiments demonstrated the effectiveness of the modified model, which can predict unexpected behaviors, and evaluate and enhance the stability during path planning. This method can be implemented for the dexterous and stable operation of magnetic catheter robots, which is significant for medical applications.

Экспериментальные исследования подвесного точечного тросового параллельного робота

A prototype of a suspended point cable parallel robot with a control interface and three translational degrees of freedom was designed and manufactured. The prototype of a point cable parallel robot consists of a metal frame in the shape of a rectangular parallelepiped; winches with servo motors are located near the base of each vertical post, designed for winding (or winding) cables. The other ends of the cables passing through the pulleys are connected to the working body. The direction of rotation, as well as the rotation speed of winches with servomotors, are set by a control unit that has three translational degrees of freedom. Four force sensors were used to determine the tension in the cables of the suspended point cable parallel robot. The force sensors are connected to the ZET 058 strain gauge measuring system, which, together with the ZETLAB TENZO software, allows you to collect information in real time through four channels simultaneously. Experimental studies of a prototype of a suspended point-cable parallel robot were carried out by drawing a circle trajectory with an end-effector. It has been established that the deviation of the trajectory from an ideal circle occurs as a result of tension fluctuations in the cables. Experimental graphs of the tension in the cables were obtained using force sensors connected to the ZET 058 strain gauge measuring system. It was established that in order to increase the accuracy of the operation of the suspended point cable parallel robot, constant monitoring of the tension of its cables is necessary.

Robotic Fast Dual-Arm Patch Clamp System for Mechanosensitive Excitability Research of Neurons.

A robotic fast dual-arm patch clamp system with controllable mechanical stimulation is proposed in this paper for mechanosensitive excitability research of neurons in brain slice. First, a kinematic model of a dual-arm patch clamp system combined with Monte Carlo method is developed to calculate the workspaces of recording micropipette and stimulation micropipette, and optimize the length of end effector for reducing collision incidences during operation. Then, a quantitative stimulation method to cells using one micropipette is developed based on pressing depth control. Finally, a fast robotic dual-arm patch clamp operation process is proposed based on a three-stage motion control of dual micropipettes to approach target cells and form whole-cell recording with quantitative mechanical stimulation. Experimental results on 50 pyramidal neurons in the primary visual cortex of mouse brain slices demonstrate that this system achieves a threefold throughput with a 37% improvement in the success rate of the contact process and a 42% improvement in the success rate of whole-cell recording in comparison to manual operation. With these advantages, a mechanical stimulation-regulated increase in neuron excitability is observed in primary visual cortex. The experimental results also show that the sodium ion current may be more sensitive to mechanical stimulation than potassium ion current. Our system significantly improves the efficiency of mechanical stimulation induced excitability research of neurons in brain slices. Our methods have the potential to investigate pathological and pathogenic mechanisms of mechanosensitive ion channel dysfunction-induced diseases in the future.

Ultrasound-Guided Robotic Autonomous Operation Based on Real-Time Deformation Tracking and Prediction

Ultrasound-Guided Robotic Autonomous Operation Based on Real-Time Deformation Tracking and Prediction

Iteratively Calibratable Network for Reliable EEG-Based Robotic Arm Control.

Robotic arms are increasingly being utilized in shared workspaces, which necessitates the accurate interpretation of human intentions for both efficiency and safety. Electroencephalogram (EEG) signals, commonly employed to measure brain activity, offer a direct communication channel between humans and robotic arms. However, the ambiguous and unstable characteristics of EEG signals, coupled with their widespread distribution, make it challenging to collect sufficient data and hinder the calibration performance for new signals, thereby reducing the reliability of EEG-based applications. To address these issues, this study proposes an iteratively calibratable network aimed at enhancing the reliability and efficiency of EEG-based robotic arm control systems. The proposed method integrates feature inputs with network expansion techniques. This integration allows a network trained on an extensive initial dataset to adapt effectively to new users during calibration. Additionally, our approach combines motor imagery and speech imagery datasets to increase not only its intuitiveness but also the number of command classes. The evaluation is conducted in a pseudo-online manner, with a robotic arm operating in real-time to collect data, which is then analyzed offline. The evaluation results demonstrated that the proposed method outperformed the comparison group in 10 sessions and demonstrated competitive results when the two paradigms were combined. Therefore, it was confirmed that the network can be calibrated and personalized using only the new data from new users.

A Robotic Teleoperation System Enhanced by Augmented Reality for Natural Human–Robot Interaction

Telekinesis, as commonly portrayed in science fiction literature and cinema, is a super power wherein users control and manipulate objects absent in physical interaction. In real world, enhancing human–robot interaction needs the synthesis of human intuitive processes with robotic arm. This paper introduces a robotic teleoperation system achieving the essence of telekinetic operations, combining the profound capabilities of augmented reality (AR) with the robotic arm operations. Utilizing AR, the proposed methodology offers operators with a visual feedback, facilitating a level of control surpassing the capacities of natural interfaces. By using AR-driven visual recognition, this system achieves operations in a virtual environment, subsequently actualized in the real world through the robotic arm. Through multiple experiments, we found that the system has a small margin of error in telekinesis operations, meeting the needs of remote operation. Furthermore, our system can operate on objects in the real world. These experiments underscore the capability of the remote control system to assist humans in accomplishing a wider range of tasks through the integration of AR and robotic arms, providing a natural human–robot interaction approach.

Methods of automated detection of travel points when training a collaborative robot

An algorithm has been developed and implemented in this paper, which allows automating the process of forming and controlling scenarios for the movement of a collaborative robot (“Cobot”) through a database of points without specific interfaces, services, and software tools characteristic of each Cobot model. The unification of the developed single graphical interface is achieved by automating the work with Cobot controllers through specialised structured file formats and Robot Operation System (ROS), and by automatically detecting marks as movement points in the image received from the stereo camera using neural network-based models and image processing techniques. Research based on a series of experiments ensured the selection of the most effective image processing method and neural network model in terms of accuracy, speed, resource consumption. The approach formalised in the paper and the graphical interface allowed to implement a classical set of industrial tasks of Cobot motion control.

Model Predictive Control based Coordinated Control for Free-Flying Space Manipulator Systems

Space Manipulator Systems (SMS) used in On-Orbit Servicing (OOS) missions are over-actuated, and safety critical systems that perform complex tasks in space. We propose a coordinated control for a free-flying SMS, which is based on a cascaded output tracking nonlinear model predictive control, to simultaneously control the spacecraft-base and manipulator of the SMS. The output tracks a given end-effector pose trajectory, and by using an artificial reference the controller stays feasible even for an unreachable trajectory. Moreover, the systems over-actuation is optimally resolved, such that the base of the SMS can be utilized to generate an unrestricted workspace for robotic operations. Throughout the entire mission, critical constraints related to system and safety, including collision avoidance, field of view of sensor devices, joint limitations, and actuator saturations, are integrated as hard constraints. Simulations of the most challenging mission phases, such as the approach and detumbling of a target spacecraft, as well as an escape maneuver verify the performance, reliability, and flexibility of our coordinated control concept.

How does Wi-Fi 6 fare? An industrial outdoor robotic scenario

Wi-Fi is a standard off-the-shelf solution for industrial robotics. The IEEE 802.11ax amendment extends it to support the 6 GHz band, 160 MHz bandwidth and bit-rates up to 9.6 Gbps. In this article, we evaluate the performance of Wi-Fi 6 compared to Wi-Fi 5 and Wi-Fi 4. We select 9 physical layers (PHYs) representing different Wi-Fi generations and we evaluate their performance in an industrial shipyard in the presence of high radio frequency interference and metallic obstructions. We deploy setup of a robotic station (STA) and a controller STA with three applications running in parallel: a robotic STA is sending a high throughput stream to a controller, a Robotic Operating System (ROS) application is sending time-critical control commands to the robotic STA, Precision Time Protocol (PTP) keeps synchronizing the clocks between both STAs. We evaluate the performance in terms of three Key Performance Indicators (KPIs): streaming throughput, IP-level delay using PTP, and Application-level delay of ROS control packets. The networks are run in: Short range Line of Sight (LoS), Medium range LoS, Long range None-LoS (NLoS), and Long range mixed settings. We note that depending on the PHY configuration, an older Wi-Fi generation may outperform Wi-Fi 6. We further observe trade-offs between the different PHYs: wide channel PHYs (e.g. 160 MHz) had best throughput reaching up to 900 Mbps while PHYs while 80 MHz or 20 MHz bandwidth achieved as low as 9 ms delay. This motivates further research in multi-PHY adaptation for KPIs specific to industrial robotics.

An interactive Augmented Reality based framework assisting operators in human-robot collaborative assembly operations

This paper proposes an interactive Augmented Reality based framework assisting operators in Human-Robot Collaborative environments where resources collaboration is required for different tasks execution. Based on its integration with the shopfloor's Digital Twin and Robotic Operation System (ROS), this framework achieves the efficient HRC in multiple aspects namely: assembly guidance provision, robot controlling, interaction with gripping tools, safety awareness, resilience in case of robot actions failure, cyber security and quality inspection result awareness. The proposed framework is tested and validated in two case studies inspired from the automotive and white good industries.

Robot-assisted retractor in spine surgery: Preliminary evaluation of its feasibility and two operation mode in beagles.

To assess the feasibility of the robot-assisted retractor. To compare the muscle injury of the two operation modes, intermittent retraction mode and continuous retraction mode in the robot-assisted retractor to find a better robot operation mode. A new robot-assisted retractor experimental platform was developed. Three incisions were made on the backs of three beagles. The robot-assisted retractor was used to retract the muscle on both sides of the incisions in intermittent retraction mode and continuous retraction mode, and the operation of the robot system was observed. The muscle samples were stained with hematoxylin-eosin (HE) to observe the muscle injury. The difference between the muscle injuries of the two groups was statistically compared using paired t test. The robot-assisted retractor can precisely retract to the specified position without malfunction or dangerous actions. Histologic evaluation showed that fewer muscle injury was found in the intermittent retraction mode group of the robot-assisted retractor compared to the continuous retraction mode group. The robot-assisted retractor offers a certain degree of feasibility and safety. The robot-assisted retractor is able to effectively reduce muscle injury with the intermittent retraction mode.

Leveraging Generative AI Prompt Programming for Human-Robot Collaborative Assembly

In manufacturing, traditional robotic programming methodologies have often been focused on independent operation, offering limited capabilities for seamless human-robot collaboration. This paper introduces a paradigm shift in collaborative production systems by leveraging generative artificial intelligence (AI), specifically large language models (LLMs). Contrary to traditional methods that rely on pre-defined assembly instructions, this paper introduces a novel framework employing primitive knowledge of the production process, including product design and required assembly steps. By integrating LLMs and a behavior tree-based system control, this approach enables programmers to rapidly deploy collaborative assembly procedures by expediting the programming of robotic operations. The system also incorporates Natural Language Processing (NLP) technologies, which facilitate real-time alterations in assembly steps, leading to reduced overall production time. The framework’s behavior tree-based control architecture allows for dynamic adaptability, offering optimized solutions across a range of assembly scenarios. The results of the framework’s deployment suggest that this innovative programming paradigm significantly enhances both the adaptability and efficiency of collaborative manufacturing settings.

Using Visual Odometry to Determine the Position of a UAV

We discuss the main concepts and problems related to navigating an unmanned aerial vehicle (UAV) in a three-dimensional space via visual odometry. In the nearest future, GMSS-free UAV navigation will embody a critical part of autonomous navigation systems, with information from on-board cameras enabling the user to estimate the UAV’s movement and position. In the given context, this paper presents different types of visual odometry, sensors for visual odometry, implementation components, and application scenarios. To allow the development and innovations, we utilize the widely used Robotic Operating System (ROS).