Robot Operating Research Articles

Robotics in Pediatric Urology: A Review

BackgroundRecent technological advances have enabled pediatric urologist to push the envelope of minimally invasive techniques. Recently, implementing robots into laparoscopic procedures has enabled the mitigation of certain drawbacks, opening the door for the implementation of minimally invasive techniques. The purpose of this review is to identify the current state surrounding robotic surgery and explore future directions for the field. MethodsThe authors undertook a literature search to explore the current state of robotic surgery in pediatric procedures. ResultsWhile operative times are longer for robotic surgery, robotic surgery operations have reported shorter post-operative hospital stays, lower pain medication use, and similar success rates relative to open procedures. New advances in training may enable improved training paradigms for novice surgeons, decreasing future operative times. ConclusionEvidence from the field suggests robotic surgery is an optimal alternative to open procedures, conferring advantages to both the patient and surgeon.

Software Architecture Design of ASV Rybitwa: Development of an Autonomous Surface Vehicle for Dynamic Navigation and Task Execution

Abstract This paper introduces the software design of an Autonomous Surface Vehicle (ASV) named ASV Rybitwa. It was designed as an easily transportable platform capable of autonomous navigation and performing tasks during the RoboBoat 2024 competition, with emphasis on its modularity, generality and extendibility. The paper presents a software architecture for a small autonomous surface vehicle using novel tools that provide easy system integration and expansion. To meet such requirements, a generic control box connecting an autopilot (PX4) and a onboard computer (NVIDIA Jetson Nano) was developed so as to host the Robot Operating System 2 (ROS2) and to interact with sensors and actuators. Computer vision from three Oak-D cameras, equipped with AI algorithm support (YOLOv8), was used for navigation and obstacle avoidance. The decision-making system was designed using behavioural trees and computer vision, allowing the vehicle’s capabilities to be adapted to the needs of the current tasks and mission. In addition, an Omni X propulsion system was proposed, providing full holonomy and enhancing dynamic positioning and navigation capabilities in complex navigational conditions. In addition this paper describes lessons learned from ASV Rybitwa’s real-world testing during the aforementioned competition.

Implementing a Vision-Based ROS Package for Reliable Part Localization and Displacement from Conveyor Belts

The use of computer vision in the industry has become fundamental, playing an essential role in areas such as quality control and inspection, object recognition/tracking, and automation. Despite this constant growth, robotic cell systems employing computer vision encounter significant challenges, such as a lack of flexibility to adapt to different tasks or types of objects, necessitating extensive adjustments each time a change is required. This highlights the importance of developing a system that can be easily reused and reconfigured to address these challenges. This paper introduces a versatile and adaptable framework that exploits Computer Vision and the Robot Operating System (ROS) to facilitate pick-and-place operations within robotic cells, offering a comprehensive solution for handling and sorting random-flow objects on conveyor belts. Designed to be easily configured and reconfigured, it accommodates ROS-compatible robotic arms and 3D vision systems, ensuring adaptability to different technological requirements and reducing deployment costs. Experimental results demonstrate the framework’s high precision and accuracy in manipulating and sorting tested objects. Thus, this framework enhances the efficiency and flexibility of industrial robotic systems, making object manipulation more adaptable for unpredictable manufacturing environments.

A ROS2-Based Gateway for Modular Hardware Usage in Heterogeneous Environments

The rise of robotics and the Internet of Things (IoT) could potentially represent a significant shift towards a more integrated and automated future, where the physical and digital domains may merge. However, the integration of these technologies presents certain challenges, including compatibility issues with existing systems and the need for greater interoperability between different devices. It would seem that the rigidity of traditional robotic designs may inadvertently make these difficulties worse, which in turn highlights the potential benefits of modular solutions. Furthermore, the mastery of new technologies may introduce additional complexity due to the varying approaches taken by robot manufacturers. In order to address these issues, this research proposes a Robot Operating System (ROS2)-based middleware, called the “ROS2-based gateway”, which aims to simplify the integration of robots in different environments. By focusing on the payload layer and enabling external communication, this middleware has the potential to enhance modularity and interoperability, thus accelerating the integration process. It offers users the option of selecting payloads and communication methods via a shell interface, which the middleware then configures, ensuring adaptability. The solution proposed in this article, based on the gateway concept, offers users and programmers the flexibility to specify which payloads they want to activate depending on the task at hand and the high-level protocols they wish to use to interact with the activated payloads. This approach allows for the optimisation of hardware resources (only the necessary payloads are activated), as well as enabling the programmer/user to utilise high-level communication protocols (such as RESTful, Kafka, etc.) to interact with the activated payloads, rather than low-level programming.

Perspectives on Intelligence in Soft Robotics

Engineers frequently aim to streamline environmental factors to facilitate the effective operation of robots. However, in nature, environmental considerations play a crucial role in shaping the embodiment of organisms. To comply robots with the complexity of real‐world environments, embedding similar intelligence is key. In the field of soft robotics, various approaches offer insight into how intelligence can be integrated into artificial agents. A discussed topic is the intricate relationship between the brain and the body at the core of intelligence in robots. The goal of this article is, therefore, to unravel the strategies to implement different types of intelligence currently adopted in soft robots. A classification is made by making a distinction between agents that adapt to their environment by 1) their adaptive shape, 2) their adaptive functionality, and 3) their adaptive mechanics. Additionally, the perspectives on intelligence based on their computational approach are distinguished: centralized computation, decentralized computation, or embedded computation. It is concluded that a tailored robotic design approach attuned to specific environmental demands is needed. To unlock the full potential of soft robots, a fresh perspective on embodied intelligence is described, so‐called mechanical intelligence, emphasizing the robot's responsiveness to changing external conditions of a real‐world environment.

ROS Gateway: Enhancing ROS Availability across Multiple Network Environments.

As the adoption of large-scale model-based AI grows, the field of robotics is undergoing significant changes. The emergence of cloud robotics, where advanced tasks are offloaded to fog or cloud servers, is gaining attention. However, the widely used Robot Operating System (ROS) does not support communication between robot software across different networks. This paper introduces ROS Gateway, a middleware designed to improve the usability and extend the communication range of ROS in multi-network environments, which is important for processing sensor data in cloud robotics. We detail its structure, protocols, and algorithms, highlighting improvements over traditional ROS configurations. The ROS Gateway efficiently handles high-volume data from advanced sensors such as depth cameras and LiDAR, ensuring reliable transmission. Based on the rosbridge protocol and implemented in Python 3, ROS Gateway is compatible with rosbridge-based tools and runs on both x86 and ARM-based Linux environments. Our experiments show that the ROS Gateway significantly improves performance metrics such as topic rate and delay compared to standard ROS setups. We also provide predictive formulas for topic receive rates to guide the design and deployment of robotic applications using ROS Gateway, supporting performance estimation and system optimization. These enhancements are essential for developing responsive and intelligent robotic systems in dynamic environments.

Zero-shot autonomous robot manipulation via natural language

Smart manufacturing revolutionizes advanced automation by leveraging deep learning and robotics technology. Nevertheless, deep learning implementation in manufacturing faces several challenges in preparing datasets, training a model, and adapting to various applications. Furthermore, programming languages are required to develop deep learning models and communicate with robots, which is not straightforward to quickly and precisely reflect human intentions in manufacturing processes. To alleviate the issue, this study proposes an autonomous zero-shot robot manipulation framework via natural language. Specifically, the Segmentation Anything Model (SAM), a promptable image segmentation model, first segments objects in images captured by an RGB-D camera, which are passed to Generative Pre-trained Transformers (GPTs), a Large Language Model (LLM), to identify and localize a designated object based on human commands given in natural language. Afterwards, the Robot Operating System (ROS) manipulates a robot to a localized position. With no additional programming or specialized skills in object localization and robot manipulation, the framework demonstrates an autonomous robot automation approach through intuitive communication between humans and machines through natural language.

Camera Calibration in High-Speed Robotic Assembly Operations

Camera Calibration in High-Speed Robotic Assembly Operations

Next‐generation human‐robot interaction with ChatGPT and robot operating system

Next‐generation human‐robot interaction with ChatGPT and robot operating system

An iterative learning-based integrated motion planning and control method for autonomous patrolling of unmanned surface vehicles

Abstract The minimization of maritime patrol durations and the optimization of patrol trajectories are still a prevailing challenge for autonomous navigation of unmanned surface vehicles (USVs). In this paper, we propose an integrated trajectory planning and control method to achieve time-optimal autonomous patrol of USVs by a data-efficient iterative learning-based predictive control algorithm. We build the optimization problem of the algorithm by introducing a local cost and a local safe constraint set for closed-loop efficient data-driven learning. To guarantee recursive feasibility, stability and convergence properties of the optimization algorithm at each iteration, the local cost and the local constraint set are designed and update iteratively using the historical vehicle states at previous iterations as a dataset. Different from traditional optimization methods, the proposed method does not require a reference path. The cost function of the optimization method decreases monotonically and converge to obtain a time-optimal control law for trajectory planning and tracking of USVs only after several iterations. The proposed approach is validated on the robotic operation system in typical maritime patrol scenarios. The results show that the proposed method implement a 5% reduction of patrol time, fewer lateral track errors, and smoother trajectories compared to the traditional MPC algorithm under water flow conditions. In addition, Enhancements in patrol operation efficiency of USVs are achieved by the algorithm, even under the constraints of varied patrol paths, attesting to its versatility.

Robot Calibration Sampling Data Optimization Method Based on Improved Robot Observability Metrics and Binary Simulated Annealing Algorithm

As the societal demand for precision in industrial robot operations increases, calibration can enhance the end-effector positioning accuracy of robots. Sampling data optimization plays an important role in improving the calibration effect. In this study, a robot calibration sampling point optimization method based on improved robot observability metrics and a Binary Simulated Annealing Algorithm is proposed. Initially, a robot kinematic model based on the Product of Exponentials (POE) model and a generalized error model is established. By utilizing the least squares method, the ideal pose transformation relationship between the robot’s base coordinate system and the laser tracker measurement coordinate system is derived, resulting in an error calibration model based on spatial single points. An improved robot observability metric combined with the Binary Simulated Annealing Algorithm (BSAA) is introduced to optimize the selection of calibration sampling data. Finally, the robot’s parameter errors are obtained using a nonlinear least squares method. Experimental results demonstrate that the average end-effector positioning error of the robot after calibration can be reduced from 0.356 mm to 0.254 mm using this method.

Deep diffusion learning of mutual-reflective structured light patterns for multi-body three-dimensional imaging

Simultaneous structured light imaging of multiple objects has become more demanding and widely in many scenarios involving robot operations in intelligent manufacturing. However, it is challenged by pattern aliasing caused by mutual reflection between high-reflective objects. To this end, we propose to learn clear fringe patterns from aliased mutual-reflective observations by diffusion models for achieving high-fidelity multi-body reconstruction in line with typical phase-shift algorithms. Regarding mutual reflection imaging as a formation of adding significant noise, we build a supervised generative learning framework based on diffusion models and then train a self-attention-based deep network with a U-Net-like skip-connected encoder-decoder architecture. We demonstrate the generalization capability of the trained model in fringe pattern recovery and its performance in phase and three-dimensional (3D) shape reconstruction. Both experimental results show that the proposed method has the expected feasibility and accuracy, heralding a promising solution for addressing the current challenge in various multi-body mutual-reflective 3D reconstruction tasks.

Endoscopic Robotic Mitral Operating Room as a Microsystem for Safety and Sustainability.

Safety and sustainability are key elements of a robotic mitral valve (MV) program at any stage of development. Challenges include the positioning of the surgeon at the robotic console, increasing patient complexity, and upstream administrative staffing difficulties. We instituted a systems approach to maximize patient safety and maintain robotic service viability. A single dedicated robotic operating room (OR) was equipped as a microsystem with team training in the operative steps, ergonomics, digital tools, and an explicit culture of safety. Outcomes of all robotic mitral procedures including concomitant procedures in the microsystem OR by a single surgeon were retrospectively reviewed. From January 2014 through December 2023, 1,529 consecutive MV patients were operated with an endoscopic robotic approach. Ten patients (0.65%) were converted to conventional approaches. Overall, 1,300 MV repairs (85%) were performed with residual MV regurgitation of none to trace in 1,205 patients (92.7%), mild in 92 patients (7.1%), and moderate in 3 patients (0.23%). MV replacements were performed in 229 patients (15%) with no paravalvular leaks. Mortality was 0.08% in the repair group and 0.87% in the replacement group. No deaths have occurred in the last 38 months. Stroke occurred in 0.31% of repair patients and 1.3% of replacement patients. One patient developed transient renal failure. Organization of the robotic OR as a microsystem is associated with surgical efficacy and very low morbidity and mortality. A comparable microsystem approach using all or select components may promote safety and sustainability for robotic MV programs at all levels.

Deep Learning-Driven Robot Arm Control Fusing Convolutional Visual Perception and Predictive Modeling for Motion Planning

The wide application of robotic technology in various industries from industrial automation to medical assistance is gradually changing our production and lifestyle, and has attracted widespread attention in many fields. However, existing robotic control systems often grapple with limited flexibility and poor adaptability to complex environments, particularly in highly dynamic operational contexts. Against this backdrop, the integration of deep learning technologies offers new possibilities in enhancing robotic perception and decision-making, especially in visual perception and motion planning. To address these challenges, we have introduced a novel robotic arm control network model, MPC-WGAN-Faster R-CNN, which combines Model Predictive Control concepts with Wasserstein Generative Adversarial Networks and Faster R-CNN visual recognition technology. This integration aims to improve the precision and adaptability of robotic arm operations in complex environments.

A Study of Library Mobile Robot Book Classification and Transportation by Integrating DA and RMM

As the complexity of modern library management tasks increases, it is difficult for traditional mobile robots to meet the task of moving and classifying books. In order to design a mobile robot that can autonomously classify and transport books, the study realizes the tasks of book classification and transportation in libraries by fusing the differential speed algorithm and the robot motion model algorithm. First, the robot operating system is utilized to scan the books, classify the books, and obtain the category information of the books. Then, the differential speed algorithm is used to control the motion of the robot to ensure that the robot can accurately transport the books to the designated location. At the same time, combined with the robot motion model algorithm, the motion trajectory of the robot is planned to ensure that the robot can avoid obstacles and stably complete the book transportation task. Finally, the deep reinforcement learning algorithm is used to train the decision-making model of the robot to improve the intelligence level of the robot. The results of simulation experiments show that the research method has the highest accuracy, with an average accuracy of 99.98%, and the robot is able to accurately categorize the books and quickly avoid obstacles with strong stability. The results of the application experiments show that the research method has the shortest moving distance, with an average moving distance of 132 m and an average completion time of 34 seconds, which are lower than the remaining three types of robots. The research robot showed high accuracy in the task of returning books in four time periods within 10 days in the library, with an average accuracy of 99.58%. The experimental results validate the superiority of the research methodology and show that the robots are capable of accurately recognizing and classifying books and can autonomously perform transportation tasks in libraries. The research results help to improve the automation level and management efficiency of libraries and have important application value.

Review and analysis of software simulators for robotic information systems

The rapid development of educational technologies requires the active involvement of new means and effective teaching methods. One of the innovative tools of learning in the field of robotics is the use of robot simulators in the educational process, which open up many opportunities for better assimilation of knowledge and development of practical skills among students. Robot simulators are software complexes that allow you to visualize and simulate the behavior of real robots. With these tools, students can: program robots in a virtual environment, test code and debug without the risk of damaging real equipment; to study robotics and the principles of robot operation, exploring different types of sensors, actuators and algorithms of robot behavior; to solve problems in various fields, such as production automation, logistics, construction, etc.; practices your skills in a safe and controlled environment. Among the benefits of using robotic simulators are next: availability for a large number of students, as simulators are much cheaper than real jobs; safety, as the virtual environment eliminates the risks associated with working with real robots; flexibility that allows students to work at their own pace, explore different scenarios and experiment with different parameters; interactivity with a realistic 3D-environment and visualization. This work presents an overview and analysis of software that allows modeling robot control systems and simulating their operation for various algorithms. A review and analysis of the proposed software tools for simulating robotic systems, which are used to teach bachelor's and master's degree students in the specialties of "Computer Science", "Information Systems", "Industrial Mechanical Engineering" and "Applied Mechanics" and have proven their effectiveness and effectiveness in distance learning at the Kyiv National University of Construction and Architecture.

Compact convolutional transformers- generative adversarial network for compound fault diagnosis of industrial robot

The safe operation of Industrial robots is a major concern in intelligent manufacturing. Accurate compound fault diagnosis is essential to the safe operation of industrial robots, while it is challenging to achieve since the compound fault samples are hard to be collected. Generative adversarial network (GAN) is a useful tool for addressing the data imbalance issue. However, the computation efficiency of GAN in addressing the data imbalance issue has not been investigated. Hence, this study proposes a lightweight GAN named compact convolutional Transformers-GAN (CCT-GAN) to alleviate the data imbalance issue in compound fault diagnosis modelling. Firstly, the feedback current signals collected from the industrial robot are transformed into time-frequency images via continuous wavelet transformation (CWT). Secondly, CCT-GAN is designed to achieve high-quality fake data generation and compound fault diagnosis modelling without large computational costs. Thirdly, the relation between a single fault and the compound fault is considered in the compound fault diagnosis modelling via multi-hot representation to alleviate the data imbalance issue. An experimental study based on the real-world compound fault dataset of industrial robots reveals that the proposed CCT-GAN shows merits in compound fault diagnosis modelling in comparison with the prevailing algorithms. The results indicate that CCT-GAN can performance of compound fault diagnosis when only 100 data samples from each compound fault category are available.

Classification of infection grade for anthracnose in mango leaves under complex background based on CBAM-DBIRNet

Anthracnose is one of the most harmful fungal diseases in mango orchards. Accurate and rapid detection of anthracnose infection grade for mango leaves in complex natural environments is a prerequisite of precise variable pesticide spraying by robots in intelligent orchards. In this paper, A lightweight double branch inverted residual attention network (CBAM-DBIRNet) model was proposed by combining the inverted residual network and the attention mechanism, which was used to detect the infection grade of anthracnose for mango leaves in natural environment. The model designed two parallel residual branch structures to extract multi-scale features of mango leaves with different infection grades through different dilation coefficients. The inverted residual structure and depthwise separable convolution were used to reduce the complexity of the model. The CBAM attention module was integrated into the model to suppress the interference of complex background information. The mango leaves collected in the natural orchard were used for experiments. The results showed that the accuracy, recall and F1 of the CBAM-DBIRNet model for the classification of anthracnose infection grade were 98.42 %, 97.80 % and 97.89 %, respectively. The model size and parameter were 0.64 MB and 0.15 M, respectively. This study provides technical support for smart orchard management and robotic precision variable application operations.

Design and Validation of an Ambulatory User Support Gait Rehabilitation Robot: NIMBLE

Relearning to walk requires progressive training in real scenarios—overground—along with assistance in basic tasks, such as balancing. In addition, user ability must be maximized through compliant robotic assistance as needed. Despite decades of research, gait rehabilitation robotic devices yield controversial results. This article presents the conceptual design of a novel walking assistance and rehabilitation robot, the NIMBLE robot, aimed at providing ambulatory, bodyweight-supported gait training, assisting the user’s center of mass trajectory to aid weight transfer and dynamic balance during walking. NIMBLE consists of a robotic mobile frame, a partial bodyweight support (PBWS) system, an ambulatory lower-limb exoskeleton (Exo-H3) and a cable-driven pelvis-assisting robot. Designed as a modular structure, it differentiates hierarchical communication levels through a Robot Operating System (ROS) 2 network. We present the mechatronic design and experimental results assessing the impact of the mechatronic coupling between the robotic modules on the walking kinematics and the frame movement control performance. The robotic frame hardly affects the walking kinematics up to 2 degrees in both the sagittal and frontal planes, making it feasible for lateral balance and weight translation training. Moreover, it successfully tracks and follows user trajectories. The NIMBLE robotic frame assessment shows promising results for ambulatory gait rehabilitation.

Investigating Human-Robot Interactions with a Quadruped Robot

Robots are becoming more prevalent in our daily lives. As robots become more common in society, we must understand how people react and interact with them. This study leverages Boston Dynamics' Spot robot to examine human responses to quadruped robots. Previous studies had Spot walking alongside participants at fixed speeds, independent of their gait. Improving upon this method, I developed a program that enables Spot to detect and follow fiducial markers (AprilTags). Using the Robot Operating System (ROS), the depth and position of the tag within the camera’s frame were extracted and translated into motion commands for the robot. The tag is attached to an apparatus on the hip that extends the tag near the subject’s knee, allowing Spot to track the participant’s trunk speed and maintain a consistent distance, offering a more adaptive walking experience. The program allows for variations in the acceleration, height, and camera selection (front/back, left/right) through a custom Graphical User Interface (GUI), allowing for diverse trial conditions. Extensive testing identified considerations to optimize the walking experience. Factors included gradual acceleration to allow Spot to react and maintain detections, and the set requirement to run only a single camera at a time for optimized processing. Additionally, keeping the tag unobstructed and square to the camera is critical to avoid failures in detection. Overall, Spot's new capabilities will assist in further studies of human and robot interactions.