Robot Operating System Research Articles

CNN-Based Multi-Object Detection and Segmentation in 3D LiDAR Data for Dynamic Industrial Environments

Autonomous navigation in dynamic environments presents a significant challenge for mobile robotic systems. This paper proposes a novel approach utilizing Convolutional Neural Networks (CNNs) for multi-object detection in 3D space and 2D segmentation using bird’s eye view (BEV) maps derived from 3D Light Detection and Ranging (LiDAR) data. Our method aims to enable mobile robots to localize movable objects and their occupancy, which is crucial for safe and efficient navigation. To address the scarcity of labeled real-world datasets, a synthetic dataset based on a simulation environment is generated to train and evaluate our model. Additionally, we employ a subset of the NVIDIA r2b dataset for evaluation in the real world. Furthermore, we integrate our CNN-based detection and segmentation model into a Robot Operating System 2 (ROS2) framework, facilitating communication between mobile robots and a centralized node for data aggregation and map creation. Our experimental results demonstrate promising performance, showcasing the potential applicability of our approach in future assembly systems. While further validation with real-world data is warranted, our work contributes to advancing perception systems by proposing a solution for multi-source, multi-object tracking and mapping.

Comparison of Collision Avoidance Algorithms for Unmanned Surface Vehicle Through Free-Running Test: Collision Risk Index, Artificial Potential Field, and Safety Zone

This paper details the development of a collision avoidance algorithm for unmanned surface vehicles (USVs) and its validation using free-running tests. The USV, designed as a catamaran, incorporates a variety of sensors for its guidance, navigation, and control system. It performs turning maneuvers using thrusters positioned on the port and starboard sides. The robot operating system is used to streamline communication, transmitting data such as position, orientation, and situational information from diverse sensors. Using the collision risk index (CRI) method, the algorithm calculates risk based on the distance to obstacles and the angle to the desired waypoint, directing the USV on a path with minimized risk. Noise within the data captured by the two-dimensional light detection and ranging system is filtered out using the k-dimensional tree and Euclidean distance methods, ensuring single obstacles are distinctly identified. To assess the efficacy of the CRI-based collision avoidance algorithm, it was benchmarked against other algorithms rooted in the artificial potential field and safety zone methods within an artificial tank setting. The results highlight the CRI method’s superior time efficiency and optimality in comparison to its counterparts.

Teleoperation system for multiple robots with intuitive hand recognition interface

Robotic teleoperation is essential for hazardous environments where human safety is at risk. However, efficient and intuitive human–machine interaction for multi-robot systems remains challenging. This article aims to demonstrate a robotic teleoperation system, denominated AutoNav, centered around autonomous navigation and gesture commands interpreted through computer vision. The central focus is on recognizing the palm of the hand as a control interface to facilitate human–machine interaction in the context of multi-robots. The MediaPipe framework was integrated to implement gesture recognition from a USB camera. The system was developed using the Robot Operating System, employing a simulated environment that includes the Gazebo and RViz applications with multiple TurtleBot 3 robots. The main results show a reduction of approximately 50% in the execution time, coupled with an increase in free time during teleoperation, reaching up to 94% of the total execution time. Furthermore, there is a decrease in collisions. These results demonstrate the effectiveness and practicality of the robotic control algorithm, showcasing its promise in managing teleoperations across multi-robots. This study fills a knowledge gap by developing a hand gesture-based control interface for more efficient and safer multi-robot teleoperation. These findings enhance human–machine interaction in complex robotic operations. A video showing the system working is available at https://youtu.be/94S4nJ3IwUw.

Theory and experiment for autonomous quadrotor flight via bounded robust finite-time homogeneous sliding mode control: A roadmap to search and rescue

This paper introduces a new saturated robust control technique for quadrotor aircraft modeled by second-order Ordinary Differential Equations (ODEs), considering disturbances in the control channel (matched disturbances). The control design employs a Sliding Mode Control (SMC) approach, featuring in: (i) a novel saturated homogeneous sliding manifold, (ii) a novel tracking controller, namely, Bounded Robust Finite-Time Homogeneous Sliding Mode Control (BRFTHSMC). An Improved Fixed-time Convergent Extended State Observer (IFCESO) is incorporated into the control scheme to handle disturbances. Together the BRFTHSMC and the IFCESO establish a reliable Active Disturbance Rejection Control (ADRC) framework. The latter ensures finite-time convergence of the errors quantities to the origin along with effective disturbance rejection. Rigorous stability analysis is conducted based on Lyapunov theory. Beside control design, another distinguishing theoretical outcome of this paper in the form of Corollary is the extension of the present results to integrator-type systems (higher-order systems). The study is substantiated through MATLAB®/Simulink simulations and Robot Operating System (ROS)/Gazebo implementation, validating the theoretical foundation. Extensive experimental tests on real hardware, including attitude and Cartesian trajectory tracking under various disturbances, further affirm the theoretical findings. The synthesized control system surpasses alternative methods in terms of control signal’s boundedness, finite-time tracking stability, transient response performance, and steady-state precision. Notably, the control input circumvents singularity challenges observed in conventional SMC approaches. In addition to trajectory tracking experiments, the controller’s effectiveness is demonstrated in real-world search and rescue scenarios. Therefore, a Deep Neural Network (DNN) algorithm, based on a MS COCO-pretrained Single Shot Detector (SSD-Mobilenet-v2), is employed for a person detection mission in an unknown search area.

A Survey of Open-Source UAV Autopilots

This survey provides a comprehensive comparison of prominent open-source unmanned aerial vehicle (UAV) autopilots, focusing on their hardware compatibility, software features, and communication protocols. Additionally, it assesses the impact of these autopilots on research and education by examining their potential for integration with companion computers, compatibility with robot operating system (ROS) middleware and the MATLAB/Simulink environment, and the availability of simulation-in-the-loop (SITL) and hardware-in-the-loop (HITL) simulation tools. The paper concludes with a discussion of the advantages and disadvantages of these leading open-source autopilots.

Real-time teleoperation of magnetic force-driven microrobots with a motion model and stable haptic force feedback for micromanipulation

Microrobots powered by an external magnetic field could be used for sophisticated medical applications such as cell treatment, micromanipulation, and noninvasive surgery inside the body. Untethered microrobot applications can benefit from haptic technology and telecommunication, enabling telemedical micro-manipulation. Users can manipulate the microrobots with haptic feedback by interacting with the robot operating system remotely in such applications. Artificially created haptic forces based on wirelessly transmitted data and model-based guidance can aid human operators with haptic sensations while manipulating microrobots. The system presented here includes a haptic device and a magnetic tweezer system linked together using a network-based teleoperation method with motion models in fluids. The magnetic microrobots can be controlled remotely, and the haptic interactions with the remote environment can be felt in real time. A time-domain passivity controller is applied to overcome network delay and ensure stability of communication. This study develops and tests a motion model for microrobots and evaluates two image-based 3D tracking algorithms to improve tracking accuracy in various Newtonian fluids. Additionally, it demonstrates that microrobots can group together to transport multiple larger objects, move through microfluidic channels for detailed tasks, and use a novel method for disassembly, greatly expanding their range of use in microscale operations. Remote medical treatment in multiple locations, remote delivery of medication without the need for physical penetration of the skin, and remotely controlled cell manipulations are some of the possible uses of the proposed technology.

Colour sorting ROS-based robot evaluation under different lights and camera angles

Automated colour sorting, aided by mobile robots, is widely prevalent in the current manufacturing industry. Obstacles, such as fluctuating light conditions and camera angles, frequently hinder this procedure. Creating a colour sorting robot is a complex and time-consuming task, especially due to the vulnerability of the RGB colour space to detection errors in extreme brightness or darkness. In response to these concerns, we introduce a mobile robot that operates on the robot operating system (ROS) platform and incorporates OpenCV. This robot employs the hue, saturation, and value (HSV) colour space model for its image processing capabilities in recognising the colours and Welzl’s algorithm for the ball’s diameter estimation. The robot’s performance was assessed across various luminous fluxes and camera tilt angles. It demonstrated exceptional performance at 64 lm and a tilt angle of 40 degrees, achieving an average accuracy of 87.5% for detecting the colour of the ball, and 81.25% for determining its location based on colour. For the ball’s diameter estimation, it was found that the best estimation was received at 64 lm and 30 degrees, with both 96.32%.

Collaborative Robot Teleoperation in Mixed Reality Environment for Inspection Tasks

Abstract Collaborative robots are increasing their presence in the industrial production context due to their flexibility in the tasks they can perform. However, programming and teleoperating them could be a difficult task from the perspective of an unqualified worker. In cases where the worker has to collaborate remotely with a robotic arm, the probability of collisions between the robotic arm and the environment increase. In this work, we describe the design and implementation of a mixed-reality multimodal interface for a worker to teleoperate a robotic arm by interacting with its holographic digital twin, while remaining aware of surrounding obstacles to avoid any collisions. To teleoperate the robot, we have developed a multimodal interface with which the user can communicate naturally with voice (through a semanticsbased task-oriented dialogue system), and by performing manual actions on holographic objects. The application also provides audiovisual information to the user about the status of the collaborative robot as well as the actions commanded. We compute and command the behavior of the robotic arm with ROS (Robot Operating System) and “MoveIt!” (motion planning framework), and we represent computed trajectories and the real-time pose of the robot in the mixed reality environment.

Basketball robot object detection and distance measurement based on ROS and IBN-YOLOv5s algorithms.

With the combination of artificial intelligence and robotics technology, more and more professional robots are entering the public eye. Basketball robot competition, as a very good target system for autonomous robot research, is very suitable for conducting research on robot autonomous perception system object detection. However, traditional basketball robots have problems such as recognition difficulties, which seriously affect the recognition of robot targets and distance measurement based on recognition. To improve the performance of basketball robots in competitions, research was conducted to improve the object detection system. Firstly, a basketball robot object detection system based on robot operating system was designed. In the software layer of the object detection system, an algorithm that combines YOLOv5s and laser detection was used, and an appropriate instance batch normalization network module was introduced in the YOLOv5s algorithm to improve the model's generalization ability. The experiment outcomes indicated that the improved algorithm had intersection over union (IoU), structural information loss, ambiguity and signal-to-noise ratio of 0.96, 0.03, 0.13, and 0.98, respectively, and performed the best in the other comparison models. The recall curve area and F1 value of the improved algorithm were 0.95 and 0.9789, respectively. In the detection of basketball, volleyball, and calibration columns, the average classification accuracy of the improved model was 95.87%, and the average calibration box accuracy was 97.05%. From this, the algorithm proposed in the study has robust performance and can efficiently achieve object detection and recognition of basketball robots. The improved algorithm proposed in the study provides more reliable and rich information for the perception ability of basketball robots, as well as for their subsequent decision-making and action planning, thereby improving the overall technical level of the robots.

Optimizing Communication for a Humanoid Subs-Talker Robot Using RSSI Analysis on ROS System

This study aims to optimize communication for the humanoid "Subs-Talker" robot using Receive Signal Strength Indicator (RSSI) analysis within the Robot Operating System (ROS). The background of this research highlights the importance of stable and strong inter-robot communication in the Indonesian Humanoid Robot Soccer Contest (KRSBI-H), where robots need to function in synergy. The method involves measuring signal strength at specific distances, both with and without the use of an extender device, utilizing the InSSIDer application for RSSI data recording and analysis. The results show that using an extender improves communication stability, especially at longer distances (7.2 and 9.0 meters), although the impact is minimal at closer distances. The discussion suggests that an extender can be an effective solution for enhancing inter-robot communication quality in competitions. In conclusion, the extender significantly improves the communication range and signal quality among robots, supporting optimal performance in collaborative robot interactions within the competition arena.

Human-in-the-loop Multi-objective Bayesian Optimization for Directed Energy Deposition with in-situ monitoring

Directed Energy Deposition (DED) is a free-form metal additive manufacturing process characterized as toolless, flexible, and energy-efficient compared to traditional processes. However, it is a complex system with a highly dynamic nature that presents challenges for modeling and optimization due to its multiphysics and multiscale characteristics. Additionally, multiple factors such as different machine setups and materials require extensive testing through single-track depositions, which can be time and resource-intensive. Single-track experiments are the foundation for establishing optimal initial parameters and comprehensively characterizing bead geometry, ensuring the accuracy and efficiency of computer-aided design and process quality validation. We digitized a DED setup using the Robot Operating System (ROS 2) and employed a thermal camera for real-time monitoring and evaluation to streamline the experimentation process. With the laser power and velocity as inputs, we optimized the dimensions and stability of the melt pool and evaluated different objective functions and approaches using a Response Surface Model (RSM). The three-objective approach achieved better rewards in all iterations and, when implemented in a real setup, allowed to reduce the number of experiments and shorten setup time. Our approach can minimize waste, increase the quality and reliability of DED, and enhance and simplify human-process interaction by leveraging the collaboration between human knowledge and model predictions.

Inquiring the robot operating system community on the state of adoption of the ROS 2 robotics middleware

AbstractThe robot operating system (ROS) is a well-established and popular open-source robotics middleware used for rapid prototyping of robotic applications. However, ROS has several known weaknesses, such as lack of support for real-time systems and limitations in fully distributed deployments with multiple robots. To address these weaknesses, ROS underwent a major update and the first version of ROS 2 was released back in 2015, being lingeringly adopted by the community in recent years. Yet, long-term support for ROS 1 will come to an end on May 2025. As such, it is more important than ever to analyze and explore the features of ROS 2, and understand its technological readiness. This work focuses on the state of adoption of ROS 2. Specifically, the article presents a user experience questionnaire targeting the ROS community to understand the more eminent needs with respect to ROS 2, determine the current levels of adoption, and identify what is holding the community back from migrating their ROS 1 applications to ROS 2. Results with more than 100 participants from around the world show that the community is aware of ROS 2 and the middleware is addressing most user expectations. We unveil the main reasons given for not migrating to ROS 2, further understand the community’s views on ROS 2, and identify what is missing to anticipate ROS 1 users migrating to ROS 2 and make ROS 2 widely adopted. Moreover, we gather important impressions on the appropriateness of ROS 2 for multi-robot system use cases.

Agent path planning based on adaptive polymorphic ant colony optimization

In the path planning of intelligent agents, ant colony algorithm is a popular path solving strategy and has been widely used. However, the traditional ant colony algorithm has problems of local optimum and redundant turning points. The adaptive polymorphic ant colony algorithm greatly improves the search and convergence speed through multi-group division and cooperation mechanism, which helps to enhance the global search ability and avoid falling into the local optimal solution. This paper proposes an improved pheromone update strategy and path selection record table construction to further improve the accuracy of path planning. Finally, the path is processed by cubic B-Spline smoothing curve to effectively reduce the turning points and realize the smoothing of the path. After MATLAB and ROS (Robot Operating System)-Gazebo simulation verification, the results show that the algorithm has good feasibility in complex environments. In summary, the improved adaptive polymorphic ant colony algorithm in this paper brings significant optimization and improvement to the global search of intelligent agents.

A Real-Time Semantic Map Production System for Indoor Robot Navigation.

Although grid maps help mobile robots navigate in indoor environments, some lack semantic information that would allow the robot to perform advanced autonomous tasks. In this paper, a semantic map production system is proposed to facilitate indoor mobile robot navigation tasks. The developed system is based on the employment of LiDAR technology and a vision-based system to obtain a semantic map with rich information, and it has been validated using the robot operating system (ROS) and you only look once (YOLO) v3 object detection model in simulation experiments conducted in indoor environments, adopting low-cost, -size, and -memory computers for increased accessibility. The obtained results are efficient in terms of object recognition accuracy, object localization error, and semantic map production precision, with an average map construction accuracy of 78.86%.

ИСКУССТВЕННЫЙ ИНТЕЛЛЕКТ В ОБРАЗОВАНИИ

The article is devoted to the problem of using robots in the educational process in the training of specialists. The article describes training robots based on the Robot Operating System, i.e. based on a set of software libraries and tools that help create applications for robots. Robot Integration with ROS (Robot Operating System) is the process of combining the hardware and software components of a robot with the ROS infrastructure. This allows the robot to interact with other devices and systems, as well as use a variety of ready-made tools and libraries available within the ecosystem. Robot Integration with ROS (Robot Operating System) is the process of combining the hardware and software components of a robot with the ROS infrastructure. This allows the robot to interact with other devices and systems, as well as use a variety of ready-made tools and libraries available within the ecosystem. Project Development: Using ROS, students can develop their own robotics projects, ranging from simple mobile robots for indoor navigation to complex manipulators for performing tasks in industry. Project Development: Using ROS, students can develop their own robotics projects, ranging from simple mobile robots for indoor navigation to complex manipulators for performing tasks in industry.

Implementation of an autonomous mobile platform for agricultural tasks in corridor-like environments

AbstractThe role of autonomous vehicles (AVs) in assisting people is recognised and, therefore, is in constant development in numerous fields. Specifically, the ability of an autonomous vehicle (AV) to alleviate global stressors, such as the increased potential for food shortages and the decline in available workers for labor-intensive tasks. An area where the development of AVs are particularly prevalent is in agriculture. However, the few AVs being used in agriculture are often custom-built for specific purposes and require long development time as a result. This article aims to build and evaluate a versatile architecture for a mobile platform that is implemented using off-the-shelf components so that it can be transferred to any agricultural vehicle, thus reducing the development time. The research has involved investigating and incorporating various sensors, and also developing a common software module to perform the localisation, navigation and mapping particularly suited for corridor crop agricultural environments. This architecture has been integrated and implemented on a Yamaha golf cart, integrating it with sensors and electronics to allow a Robotic Operating System (ROS) framework to gather information and control the vehicle. As the architecture is modular in nature, it can be transferred to different customised platforms. To determine the efficacy of the mobile platform, it has undergone evaluation in simulation and in the field. The evaluation demonstrates that both mapping and navigation have satisfactory results, and the mobile platform remains within 5 mm of the specified distance when aiming to follow the row in a vineyard. The results from these experiments demonstrate the ability of the mobile platform to successfully transform a Yamaha golf cart into an autonomous agricultural vehicle.

EddyBot: A multichannel FPGA-based Eddy Current Testing mobile robot

We introduce EddyBot, a novel integration of a multi-channel FPGA-based Eddy Current Testing (ECT) scanner within a mobile robot designed for autonomous non-destructive testing (NDT). EddyBot distinguishes itself by leveraging advanced signal processing, real-time robotics control, and efficient data synchronization to significantly enhance inspection accuracy and efficiency. Our contributions are (i) the deployment of a Zynq-7020 FPGA for fast data acquisition and processing, (ii) alongside the Robot Operating System (ROS) for precise and real-time control with Kalman filtering (KF) sensor fusion for pose estimation. (iii) We further optimize defect detection and localization through a novel data synchronization approach, correlating the robot’s pose with ECT signals to a mean synchronization delay of only 11.77 ms. This allows for precise localization of defects within 0.18 cm, improving the overall reliability. The robot’s performance was evaluated on aluminum and ferromagnetic specimens, highlighting the system’s capability to detect defects with high accuracy.

AN IMPROVED COVERAGE PATH PLANNING FOR SERVICE ROBOTS BASED ON BACKTRACKING METHOD

Service robots are of paramount importance in both industrial and residential environments. Nevertheless, these robots frequently encounter obstacles in the form of perilous elements present in their surroundings, which may result in their malfunction and hinder their optimal performance. Hence, autonomous robot applications heavily rely on coverage path planning algorithms, which facilitate the execution of area coverage operations in a streamlined and economical manner. The article presents a coverage path planning algorithm for service robots that is based on a backtracking procedure. The proposed method attempts to strike a balance between the safety of the cleaning automaton and the extent of coverage. Following straight-forward instructions, the mobile robot (MR) initially generates a spiral or zigzag trajectory. It examines each position for possible backtracking locations. When obstacles and visited positions obstruct the path, the improved A* algorithm is utilized to return to known unvisited regions. Subsequently, the robot proceeds to sweep the regions until no remaining back-tracking sites remain. Simulation of the method on the Robot Operating System (ROS) verifies that it is possible to restrict the movement of high-risk robots.

Software Architecture Design of ASV Rybitwa: Development of an Autonomous Surface Vehicle for Dynamic Navigation and Task Execution

Abstract This paper introduces the software design of an Autonomous Surface Vehicle (ASV) named ASV Rybitwa. It was designed as an easily transportable platform capable of autonomous navigation and performing tasks during the RoboBoat 2024 competition, with emphasis on its modularity, generality and extendibility. The paper presents a software architecture for a small autonomous surface vehicle using novel tools that provide easy system integration and expansion. To meet such requirements, a generic control box connecting an autopilot (PX4) and a onboard computer (NVIDIA Jetson Nano) was developed so as to host the Robot Operating System 2 (ROS2) and to interact with sensors and actuators. Computer vision from three Oak-D cameras, equipped with AI algorithm support (YOLOv8), was used for navigation and obstacle avoidance. The decision-making system was designed using behavioural trees and computer vision, allowing the vehicle’s capabilities to be adapted to the needs of the current tasks and mission. In addition, an Omni X propulsion system was proposed, providing full holonomy and enhancing dynamic positioning and navigation capabilities in complex navigational conditions. In addition this paper describes lessons learned from ASV Rybitwa’s real-world testing during the aforementioned competition.

Implementing a Vision-Based ROS Package for Reliable Part Localization and Displacement from Conveyor Belts

The use of computer vision in the industry has become fundamental, playing an essential role in areas such as quality control and inspection, object recognition/tracking, and automation. Despite this constant growth, robotic cell systems employing computer vision encounter significant challenges, such as a lack of flexibility to adapt to different tasks or types of objects, necessitating extensive adjustments each time a change is required. This highlights the importance of developing a system that can be easily reused and reconfigured to address these challenges. This paper introduces a versatile and adaptable framework that exploits Computer Vision and the Robot Operating System (ROS) to facilitate pick-and-place operations within robotic cells, offering a comprehensive solution for handling and sorting random-flow objects on conveyor belts. Designed to be easily configured and reconfigured, it accommodates ROS-compatible robotic arms and 3D vision systems, ensuring adaptability to different technological requirements and reducing deployment costs. Experimental results demonstrate the framework’s high precision and accuracy in manipulating and sorting tested objects. Thus, this framework enhances the efficiency and flexibility of industrial robotic systems, making object manipulation more adaptable for unpredictable manufacturing environments.