Gazebo Environment Research Articles

On the Development of an Acoustic Image Dataset for Unexploded Ordnance Classification Using Front-Looking Sonar and Transfer Learning Methods.

This research aimed to develop a dataset of acoustic images recorded by a forward-looking sonar mounted on an underwater vehicle, enabling the classification of unexploded ordnances (UXOs) and objects other than unexploded ordnance (nonUXOs). The dataset was obtained using digital twin simulations performed in the Gazebo environment utilizing plugins developed within the DAVE project. It consists of 69,444 sample images of 512 × 399 resolution organized in two classes annotated as UXO and nonUXO. The obtained dataset was then evaluated by state-of-the-art image classification methods using off-the-shelf models and transfer learning techniques. The research included VGG16, ResNet34, ResNet50, ViT, RegNet, and Swin Transformer. Its goal was to define a base rate for the development of other specialized machine learning models. Neural network experiments comprised two stages-pre-training of only the final layers and pre-training of the entire network. The experiments revealed that to obtain high accuracy, it is required to pre-train the entire network, under which condition, all the models achieved comparable performance, reaching 98% balanced accuracy. Surprisingly, the highest accuracy was obtained by the VGG model.

Sterowanie autonomicznym bezzałogowym statkiem powietrznym z wykorzystaniem uczenia przez wzmacnianie

Reinforcement learning is of increasing importance in the field of robot control and simulation plays a key role in this process. In the unmanned aerial vehicles (UAVs, drones), there is also an increase in the number of published scientific papers involving this approach. In this work, an autonomous drone control system was prepared to fly forward (according to its coordinates system) and pass the trees encountered in the forest based on the data from a rotating LiDAR sensor. The Proximal Policy Optimization (PPO) algorithm, an example of reinforcement learning (RL), was used to prepare it. A custom simulator in the Python language was developed for this purpose. The Gazebo environment, integrated with the Robot Operating System (ROS), was also used to test the resulting control algorithm. Finally, the prepared solution was implemented in the Nvidia Jetson Nano eGPU and verified in the real tests scenarios. During them, the drone successfully completed the set task and was able to repeatable avoid trees and fly through the forest.

Stabilization and control on a pipe-rack of a wheeled mobile manipulator with a snake-like arm

Autonomous inspection and maintenance tasks with robots in oil and gas refineries require moving along pipelines and manipulation dexterity in cluttered environments. This paper investigates the problem of controlling a wheeled mobile manipulator endowed with a snake-like arm to inspect the structures while stabilizing the supporting pipe. A model predictive control approach stabilizes the wheeled robot on the pipe. When the wheel torques saturate, the stabilization task leverages the resulting propagating force on the wheeled robot given by the snake-like arm’s dynamics. The significant number of degrees of freedom given by the snake-like arm allows a prioritized redundancy resolution scheme with hybrid motion/force tracking to inspect the same and adjacent pipes while avoiding self-collisions and environmental impacts. Simulations in the realistic Gazebo environment validate the achieved preliminary results.

A Novel Autonomous Landing Method for Flying–Walking Power Line Inspection Robots Based on Prior Structure Data

Hybrid inspection robots have been attracting increasing interest in recent years, and are suitable for inspecting long-distance overhead power transmission lines (OPTLs), combining the advantages of flying robots (e.g., UAVs) and climbing robots (e.g., multiple-arm robots). Due to the complex work conditions (e.g., power line slopes, complex backgrounds, wind interference), landing on OPTL is one of the most difficult challenges faced by hybrid inspection robots. To address this problem, this study proposes a novel autonomous landing method for a developed flying–walking power line inspection robot (FPLIR) based on prior structure data. The proposed method includes three main steps: (1) A color image of the target power line is segmented using a real-time semantic segmentation network, fusing the depth image to estimate the position of the power line. (2) The safe landing area (SLA) is determined using prior structure data, applying the trajectory planning method with geometric constraints to generate the dynamic landing trajectory. (3) The landing trajectory is tracked using real-time model predictive control (MPC), controlling FPLIR to land on the OPTL. The feasibility of the proposed method was verified in the ROS Gazebo environment. The RMSE values of the position along three axes were 0.1205,0.0976 and 0.0953, respectively, while the RMSE values of the velocity along these axes were 0.0426, 0.0345 and 0.0781. Additionally, experiments in a real environment using FPLIR were performed to verify the validity of the proposed method. The experimental results showed that the errors of position and velocity for the FPLIR landing on the lines were 6.18×10−2 m and 2.16×10−2 m/s. The simulation results as well as the experimental findings both satisfy the practical requirements. The proposed method provides a foundation for the intelligent inspection of OPTL in the future.

Robot operating system based autonomous navigation platform with human robot interaction

In emerging technologies, indoor service robots are playing a vital role for people who are physically challenged and visually impaired. The service robots are efficient and beneficial for people to overcome the challenges faced during their regular chores. This paper proposes the implementation of autonomous navigation platforms with human-robot interaction which can be used in service robots to avoid the difficulties faced in daily activities. We used the robot operating system (ROS) framework for the implementation of algorithms used in auto navigation, speech processing and recognition, and object detection and recognition. A suitable robot model was designed and tested in the Gazebo environment to evaluate the algorithms. The confusion matrix that was created from 125 different cases points to the decent correctness of the model.

English

This paper presents the research and construction of a motion tracing control system for omnidirectional mobile robots based on reinforcement learning techniques in automatic control. The process of controlling a mobile robot in a flat environment with definite and unknown obstacles, taking into account the nonlinear factor of interference. Research and application of programming tools are operating systems for mobile robots (Robot Operating System - ROS). From updated information on maps, operating environment, robot control position, and obstacle identification (SLAM) to calculate the movement trajectory of a three-wheeled omnidirectional mobile robot. The positioning system calculates the orbital tracking for the robot based on the Q-learning algorithm. The results of simulation research in the Gazebo environment and running tests on real Turtlebot mobile robots have shown the practical effectiveness of the research problem of tracking motion tracking and intelligent navigation for mobile robots.

Using optical sensors for industrial robot-human interactions in a Gazebo environment.

Using optical sensors for industrial robot-human interactions in a Gazebo environment.

Error State Extended Kalman Filter Localization for Underground Mining Environments

The article addresses the issue of mobile robotic platform positioning in GNSS-denied environments in real-time. The proposed system relies on fusing data from an Inertial Measurement Unit (IMU), magnetometer, and encoders. To get symmetrical error gauss distribution for the measurement model and achieve better performance, the Error-state Extended Kalman Filter (ES EKF) is chosen. There are two stages of vector state determination: vector state propagation based on accelerometer and gyroscope data and correction by measurements from additional sensors. The error state vector is composed of the velocities along the x and y axes generated by combining encoder data and the orientation of the magnetometer around the axis z. The orientation angle is obtained from the magnetometer directly. The key feature of the algorithm is the IMU measurements’ isolation from additional sensor data, with its further summation in the correction step. Validation is performed by a simulation in the ROS (Robot Operating System) and the Gazebo environment on the grounds of the developed mathematical model. Trajectories for the ES EKF, Extended Kalman Filter (EKF), and Unscented Kalman Filter (UKF) algorithms are obtained. Absolute position errors for all trajectories are calculated with an EVO package. It is shown that using the simplified version of IMU’s error equations allows for the achievement of comparable position errors for the proposed algorithm, EKF and UKF.

Development of a somatosensory controller for positioning a manipulator device for picking apples

The aim of the study is to develop a somatosensory controller for remote control of a manipulator for picking apples using resistive sensors, a gyroscope and an accelerometer. A computer simulation of the process of picking fruits from a tree crown was carried out in the Gazebo simulator using a somatosensory controller. An environment has been created, including a model of a manipulator for picking apple fruits and a fruit tree, and the properties of the environment have been set. To conduct a comparative experiment to evaluate the results of computer simulation of the process of picking apple fruits using a self-sensor controller and a manufactured sample of a manipulator, artificial models of trees with an identical crown and the location of fruits in the crown at a distance of no more than 1,5 meters from the zero point of the manipulator were used. According to the results of the research, it was found that the average time for picking one apple fruit by a manufactured manipulator in an automated mode is 12,9 seconds, the minimum time was 9,4 seconds, the maximum 15,9 seconds. Gazebo environment using a self-touch controller was 11,7 seconds. The minimum time was 8 seconds, the maximum 14,9 seconds. The development of digital intelligent systems, telematic services will ensure the widespread use of the proposed method for monitoring and managing the group work of robotic tools in the cultivation of fruit and berry crops.

A Multi-Agent System Using Decentralized Decision-Making Techniques for Area Surveillance and Intruder Monitoring

A decentralized swarm of quadcopters designed for monitoring an open area and detecting intruders is proposed. The system is designed to be scalable and robust. The most important aspect of the system is the swarm intelligent decision-making process that was developed. The rest of the algorithms essential for the system to be completed are also described. The designed algorithms were developed using ROS and tested with SITL simulations in the GAZEBO environment. The proposed approach was tested against two other similar surveilling swarms and one approach using static cameras. The addition of the real-time decision-making capability offers the swarm a clear advantage over similar systems, as depicted in the simulation results.

Robust Neural Network Consensus for Multiagent UASs Based on Weights’ Estimation Error

We propose a neural network consensus strategy to solve the leader–follower problem for multiple-rotorcraft unmanned aircraft systems (UASs), where the goal of this work was to improve the learning based on a set of auxiliary variables and first-order filters to obtain the estimation error of the neural weights and to introduce this error information in the update laws. The stability proof was conducted based on Lyapunov’s theory, where we concluded that the formation errors and neural weights’ estimation error were uniformly ultimately bounded. A set of simulation results were conducted in the Gazebo environment to show the efficacy of the novel update laws for the altitude and translational dynamics of a group of UASs. The results showed the benefits and insights into the coordinated control for multiagent systems that considered the weights’ error information compared with the consensus strategy based on classical σ-modification. A comparative study with the performance index ITAE and ITSE showed that the tracking error was reduced by around 45%.

Double-Loop PID-Type Neural Network Sliding Mode Control of an Uncertain Autonomous Underwater Vehicle Model Based on a Nonlinear High-Order Observer with Unknown Disturbance

An unknown nonlinear disturbance seriously affects the trajectory tracking of autonomous underwater vehicles (AUVs). Thus, it is critical to eliminate the influence of such disturbances on AUVs. To address this problem, this paper proposes a double-loop proportional–integral–differential (PID) neural network sliding mode control (DLNNSMC). First, a double-loop PID sliding mode surface is proposed, which has a faster convergence speed than other PID sliding mode surfaces. Second, a nonlinear high-order observer and a neural network are combined to observe and compensate for the nonlinear disturbance of the AUV system. Then, the bounded stability of an AUV closed-loop system is analyzed and demonstrated using the Lyapunov method, and the time-domain method is used to verify that the velocity- and position-tracking errors of AUVs converge to zero exponentially. Finally, the radial basis function (RBF) neural network PID sliding mode control (RBFPIDSMC) and the RBF neural network PID sliding mode control (RBFPDSMC) are compared with this method in two trajectory tracking control simulation experiments. In the first experiment, the average Euclidean distance of the position-tracking error for this method was reduced by approximately 73.6% and 75.3%, respectively, compared to those for RBFPDSMC and RBFPIDSMC. In the second experiment, the average Euclidean distance of the position tracking error for this method was reduced by approximately 86.8% and 88.8%, respectively. The two experiments showed that the proposed control method has a strong anti-jamming ability and tracking effect. The simulation results obtained in the Gazebo environment validated the superiority of this method.

CACLA-Based Local Path Planner for Drones Navigating Unknown Indoor Corridors

This article presents an online local path planning approach for autonomous drone navigating a 2D plane in an unknown, indoor corridor-like environment. The proposed method utilizes a reinforcement learning approach for training a local path planner for navigation in the said environment. With a continuous actor-critic learning automaton (CACLA) applied for continuous action spaces, the proposed algorithm uses a reward structure that formulates a balancing function that gives reward based on balancing the vehicle between artificial potential hills. The drone thereby learns steering control and obstacle avoidance while maintaining a central aligned position with respect to the unknown hallways or corridors. A novel CACLA algorithm and incorporation of a special experience replay memory for the better converging tendency of drone toward the balancing point have been introduced in this article. The proposed reinforcement learning-based online local path planner has been tested on a simulated drone in Gazebo environment.

깊이 이미지를 이용한 딥러닝 기반 충돌 확률 예측 및 장애물 회피 알고리즘

This paper proposes a vision-based navigation algorithm in cluttered environments by using a convolutional neural network (CNN)-based deep learning and depth images. Existing approaches for robot navigation are developed based on obstacle detection or an accurate map to generate a path, which requires complex computation. To solve this issue, we propose a deep-learning-based navigation algorithm that does not require a preobtained map or a recognition of obstacles. In addition, by exploiting a collision probability prediction using a depth image, our method automatically adjusts the velocity command while passing the complex area, so we can improve the safety of a mobile robot. To validate the performance of our proposed method, we conduct a simulation with a mobile robot in a Gazebo environment. The simulation results show that our method outperforms the conventional avoidance algorithm.

Rail Tracking and Detection with Drone in Gazebo Environment

Rail transport is considered one of the safest modes of transport. Along with the development of technology, significant developments have been observed in railway transportation over the years. With the increasing railway line, the demand for railway transportation is increasing day by day. The number of passengers using railway transportation has also increased in this context. With this intensity, the damage to the railway line increases. In ensuring the safety of railway transport, methods based on deep learning have become important to ensure railway safety. In order for the railway line to provide healthy service, the monitoring of the railway line should be done regularly. Traditional rail monitoring services require different vehicles. Today, besides conventional vehicles, drones are used for the monitoring of the railway line. Experimenting with drones in the real environment can be difficult and costly. It is always more advantageous to run the codes that will run on the drone first in a simulation environment to save time. In this study, a drone-based system that autonomously tracks and detects railway tracks is proposed as an alternative to traditional methods. The proposed method detects the rails with the semantic segmentation method and follows the rails with its front camera. The proposed method was developed in the Gazebo environment. The general purpose of the study is to record the rail images with the drone camera that follows the railway autonomously. In this way, drone experiments to be carried out in the real railway environment will be completed in a shorter time.

Visual-Inertial Image-Odometry Network (VIIONet): A Gaussian process regression-based deep architecture proposal for UAV pose estimation

This study estimates the pose of Unmanned Aerial Vehicle (UAV) through artificial intelligence-based approaches and combines visual-inertial information in a different way than previous studies. For an effective fusion, the inertial data between both frames is normalized after denoising with the Savitzky-Golay technique and finally converted from numerical value to image. To strengthen these inertial image features with the change of motion between two frames, frames of Optical Flow (OF) are obtained and OF frames are combined with inertial images. Simultaneously, a parallel thread combines this OF frame with two consecutive raw frames. After features are extracted from inertial and camera data via Inception-v3, these features are fused and actual UAV poses are estimated via Gaussian Process Regression (GPR). Thanks to the smoothing process applied to these estimated values, a more stable pose estimation is provided. This proposed method is applied to the EuRoC dataset and our dataset produced in the Gazebo environment. The pose estimation results reveal that the proposed method has high performance compared to many previous studies.

Autonomous Cooperative Multi-Vehicle System for Interception of Aerial and Stationary Targets

This paper presents the design, development, and testing of hardware-software systems by the IISc-TCS team for Challenge 1 of the Mohamed Bin Zayed International Robotics Challenge 2020. The goal of Challenge 1 was to grab a ball suspended from a maneuvering UAV and pop balloons anchored to the ground, using suitable manipulators. The important tasks carried out to address this challenge include the design and development of a hardware system with efficient grabbing and popping mechanisms, considering the restrictions in volume and payload, design of accurate target interception algorithms using visual information suitable for outdoor environments, and development of a software architecture for dynamic, multi-agent, aerial systems performing complex tasks. In this paper, we discuss the design of a custom end-effector mounted on a single degree of freedom manipulator, and robust algorithms for the interception of targets in an uncertain environment. Vision-based guidance and tracking strategies are developed based on the concept of pursuit engagement and artificial potential function. The software architecture presented in this work develops an Operation Management System (OMS) architecture that allocates static and dynamic tasks collaboratively among multiple UAVs to perform any given task. An important aspect of this work is that all the systems developed were designed to operate in completely autonomous mode. A detailed description of the architecture along with simulations of complete challenge in the Gazebo environment and field experiment results are also included in this work. The developed hardware-software system is useful for counter-UAV systems and can be used for other applications.

Derived blockchain architecture for security-conscious data dissemination in edge-envisioned Internet of Drones ecosystem

Internet of Drones (IoD) facilitates the autonomous operations of drones into every application (warfare, surveillance, photography, etc) across the world. The transmission of data (to and fro) related to these applications occur between the drones and the other infrastructure over wireless channels that must abide to the stringent latency restrictions. However, relaying this data to the core cloud infrastructure may lead to a higher round trip delay. Thus, we utilize the cloud close to the ground, i.e., edge computing to realize an edge-envisioned IoD ecosystem. However, as this data is relayed over an open communication channel, it is often prone to different types of attacks due to it wider attack surface. Thus, we need to find a robust solution that can maintain the confidentiality, integrity, and authenticity of the data while providing desired services. Blockchain technology is capable to handle these challenges owing to the distributed ledger that store the data immutably. However, the conventional block architecture pose several challenges because of limited computational capabilities of drones. As the size of blockchain increases, the data flow also increases and so does the associated challenges. Hence, to overcome these challenges, in this work, we have proposed a derived blockchain architecture that decouples the data part (or block ledger) from the block header and shifts it to off-chain storage. In our approach, the registration of a new drone is performed to enable legitimate access control thus ensuring identity management and traceability. Further, the interactions happen in the form of transactions of the blockchain. We propose a lightweight consensus mechanism based on the stochastic selection followed by a transaction signing process to ensure that each drone is in control of its block. The proposed scheme also handles the expanding storage requirements with the help of data compression using a shrinking block mechanism. Lastly, the problem of additional delay anticipated due to drone mobility is handled using a multi-level caching mechanism. The proposed work has been validated in a simulated Gazebo environment and the results are promising in terms of different metrics. We have also provided numerical validations in context of complexity, communication overheads and computation costs.

영상센서를 이용한 실외 무인항공기의 충돌회피에 관한 연구

This study investigates real-time path planning for UAVs (Unmanned Aerial Vehicles) to avoid obstacles in an outdoor environment in a three-dimensional space. First, we implement an algorithm to convert from Octomap to a grid map to implement a global grid map. Secondly, the ESDF (Euclidean Signed-Distance Field) algorithm is integrated into the path planning obtained through the JPS (Jump Point Search) algorithm, and the B-Spline algorithm is applied to modify the path to consider the dynamic constraint of UAVs. Finally, path planning algorithms are verified in real-time through simulation and flight testing. The simulation is based on the Gazebo environment using the open-source autopilot PX4 with the ROS. A depth camera performance is compared for an outdoor flight test to select a depth camera mounted on the drone. Moreover, the practical obstacle-avoidance performance of the proposed real-time route planning in an outdoor space is confirmed through a flight test.

Digital twin for the integration of the automatic transport and manufacturing processes

Simulation techniques for time calculation in industrial transport are not currently deterministic. This supposes a time uncertainty in production planning with economic implications. The digitization promoted by Industry 4.0 allows the automation of transport in manufacturing processes. In this way, the Automated Guided Vehicles (AGVs) performance can be simulated to obtain more accurate results. The definition of a virtual simulation environment through digital twins is an incipient area of research in this field. Digital twins achieve this purpose without intervening in the physical world thus minimising costs. In order to facilitate the use of digital twins, in this paper, we implement a web-based simulation service to improve the user experience. The user will be able to pose hypotheses and visualise the execution in real-time. The techniques implemented for the simulations are based on Robot Operating System (ROS) within the Gazebo environment, IIoT communications require the Node-network programming environment which is in charge of the hyperconnectivity and the creation of the user interface.