Autonomous Navigation Research Articles

Multi-feature fusion and multi-attention deep network for enhancing road extraction in remote sensing images

ABSTRACT Road detection in remote sensing (RS) images plays a critical role in applications ranging from urban planning to autonomous navigation systems. However, accurate road extraction remains a challenging task due to the presence of textual-similar objects that can be visually confused with roads, and shadows that can obscure road features. For this regard, we present a novel end-to-end network that leverages multi-feature fusion and multi-attention. Multi-level dual residual blocks are introduced to capture multi-scale and multi-level road features. A channel attention feature fusion module fuses road features from the decoder while suppressing coarse-grain noises. Moreover, a difference channel feature fusion module filters out the interfering texture-similar features and irrelevant background features. Subsequently, a spatial channel feature enhancement module is designed to identify the mis-segmented regions and reclassify these pixels. We conducted extensive experiments on diverse datasets, demonstrating the effectiveness of our approach in improving road detection accuracy, especially in areas with shadows and textual-similar objects. The results indicate that our approach outperforms state-of-the-art methods in handling textual-similar objects and shadow-occluded roads, making it a valuable contribution to the field of road extraction in remote sensing images.

Inspiration from Visual Ecology for Advancing Multifunctional Robotic Vision Systems: Bio-inspired Electronic Eyes and Neuromorphic Image Sensors.

In robotics, particularly for autonomous navigation and human-robot collaboration, the significance of unconventional imaging techniques and efficient data processing capabilities is paramount. The unstructured environments encountered by robots, coupled with complex missions assigned to them, present numerous challenges necessitating diverse visual functionalities, and consequently, the development of multifunctional robotic vision systems has become indispensable. Meanwhile, rich diversity inherent in animal vision systems, honed over evolutionary epochs to meet their survival demands across varied habitats, serves as a profound source of inspirations. Here, recent advancements in multifunctional robotic vision systems drawing inspiration from natural ocular structures and their visual perception mechanisms are delineated. First, unique imaging functionalities of natural eyes across terrestrial, aerial, and aquatic habitats and visual signal processing mechanism of humans are explored. Then, designs and functionalities of bio-inspired electronic eyes are explored, engineered to mimic key components and underlying optical principles of natural eyes. Furthermore, neuromorphic image sensors are discussed, emulating functional properties of synapses, neurons, and retinas and thereby enhancing accuracy and efficiency of robotic vision tasks. Next, integration examples of electronic eyes with mobile robotic/biological systems are introduced. Finally, a forward-looking outlook on the development of bio-inspired electronic eyes and neuromorphic image sensors is provided.

Development of a calculation power model of a vacuum automounting device

One of the ways to improve navigation safety during water transport operation is to develop automated devices and systems that reduce the number of tasks for workers. For example, the introduction of vessels with different levels of autonomy into production activities, capable of automatically monitoring the environment or delivering cargo, helps to reduce the size of the vessel due to the absence of a residential superstructure. On inland waterways, one of the limitations of the introduction of autonomous navigation is their safe passage through shipping passages — shipping locks. Mooring a vessel in a lock chamber is very labor-intensive and is usually performed by crew members of the vessel being locked for movable (floating hoops) or fixed (berthing bollards, bollards, hoops) mooring devices, the permissible loads of which are established in regulatory documents. Over the past 10 years, the authors have not found studies describing the results of measuring the loads on mooring devices and ropes arising from the movement of the vessel in the lock chamber. Another constraint to the implementation of autonomous shipping is the lack of requirements for auto-mooring devices that ensure the retention of vessels in the lock chamber. In this regard, the purpose of this study is to compare the standard and actual values of the load on mooring eyes and ropes for the theoretical calculation of the holding forces of the vacuum automooring device model, taking into account the operating conditions in the shipping lock chamber. The measurement results presented in the article were obtained in the course of the research work “Development of scientifically based recommendations for establishing hydraulic modes for filling and emptying the chambers of locks No. 1–8 of the Federal State Budgetary Institution “Volga-Balt Administration” based on in-kind studies of the conditions of ship mooring in lock chambers” commissioned by the Federal Agency for Maritime and River Transport using a strain gauge installed in the mooring rope cut. As a result of the theoretical calculation of the holding force of the vacuum auto-mooring device, the holding forces of the vacuum gripper from the radius of one vacuum suction cup were obtained.

Map Construction and Positioning Method for LiDAR SLAM-Based Navigation of an Agricultural Field Inspection Robot

In agricultural field inspection robots, constructing accurate environmental maps and achieving precise localization are essential for effective Light Detection And Ranging (LiDAR) Simultaneous Localization And Mapping (SLAM) navigation. However, navigating in occluded environments, such as mapping distortion and substantial cumulative errors, presents challenges. Although current filter-based algorithms and graph optimization-based algorithms are exceptionally outstanding, they exhibit a high degree of complexity. This paper aims to investigate precise mapping and localization methods for robots, facilitating accurate LiDAR SLAM navigation in agricultural environments characterized by occlusions. Initially, a LiDAR SLAM point cloud mapping scheme is proposed based on the LiDAR Odometry And Mapping (LOAM) framework, tailored to the operational requirements of the robot. Then, the GNU Image Manipulation Program (GIMP) is employed for map optimization. This approach simplifies the map optimization process for autonomous navigation systems and aids in converting the Costmap. Finally, the Adaptive Monte Carlo Localization (AMCL) method is implemented for the robot’s positioning, using sensor data from the robot. Experimental results highlight that during outdoor navigation tests, when the robot operates at a speed of 1.6 m/s, the average error between the mapped values and actual measurements is 0.205 m. The results demonstrate that our method effectively prevents navigation mapping distortion and facilitates reliable robot positioning in experimental settings.

Intelligent agricultural robotic detection system for greenhouse tomato leaf diseases using soft computing techniques and deep learning

The development of soft computing methods has had a significant influence on the subject of autonomous intelligent agriculture. This paper offers a system for autonomous greenhouse navigation that employs a fuzzy control algorithm and a deep learning-based disease classification model for tomato plants, identifying illnesses using photos of tomato leaves. The primary novelty in this study is the introduction of an upgraded Deep Convolutional Generative Adversarial Network (DCGAN) that creates augmented pictures of disease tomato leaves from original genuine samples, considerably enhancing the training dataset. To find the optimum training model, four deep learning networks (VGG19, Inception-v3, DenseNet-201, and ResNet-152) were carefully compared on a dataset of nine tomato leaf disease classes. These models have validation accuracy of 92.32%, 90.83%, 96.61%, and 97.07%, respectively, when using the original PlantVillage dataset. The system then uses an enhanced dataset with ResNet-152 network design to achieve a high accuracy of 99.69%, as compared to the original dataset with ResNet-152’s accuracy of 97.07%. This improvement indicates the use of the proposed DCGAN in improving the performance of the deep learning model for greenhouse plant monitoring and disease detection. Furthermore, the proposed approach may have a broader use in various agricultural scenarios, potentially altering the field of autonomous intelligent agriculture.

Hybrid Mobile Robot Path Planning Using Safe JBS-A*B Algorithm and Improved DWA Based on Monocular Camera

This paper addresses the formidable challenge of enabling autonomous navigation in Mobile Robots (MRs), focusing on the development of advanced path planning strategies. Despite their pivotal role in diverse applications, they face challenges in dynamic settings due to limitation in existing Global Path Planning (GPP) and Local Path Planning (LPP) techniques. In response to this, we propose an innovative hybrid path planning approach that enhances the A* algorithm with a risk-aware heuristic function and integrates the Jump Point Search (JPS) technique for route optimization. Additionally, B-spline smoothing is employed for perceptually global trajectory refinement. Our approach also includes an innovative improvement to the Dynamic Window Approach (DWA) to align with the proposed enhanced A* algorithm for effective local navigation. Acknowledging the importance of high-quality input in path planning, we present substantial improvements to the IRDC-Net, a monocular-image semantic-segmentation model that we studied previously. Novel improvements include the integration of quantization and the Adam optimizer, along with the implementation of the Balanced Cross-Entropy loss function. These enhancements not only elevate the output quality of IRDC-Net but also reduce the model’s training parameters. The experimental results demonstrate the performance and viability of the proposed algorithm. Ultimately, the hybrid MR’s path planning algorithm exhibits proficiency across various tasks, particularly in addressing the challenge of evading moving obstacles to ensure the robot’s safety while adhering to the global path.

DGRO: Doppler Velocity and Gyroscope-Aided Radar Odometry

A stable and robust odometry system is essential for autonomous robot navigation. The 4D millimeter-wave radar, known for its resilience in harsh weather conditions, has attracted considerable attention. As the latest generation of FMCW radar, 4D millimeter-wave radar provides point clouds with both position and Doppler velocity information. However, the increased uncertainty and noise in 4D radar point clouds pose challenges that prevent the direct application of LiDAR-based SLAM algorithms. To address this, we propose a SLAM framework that fuses 4D radar data with gyroscope readings using graph optimization techniques. Initially, Doppler velocity is employed to estimate the radar’s ego velocity, with dynamic points being removed accordingly. Building on this, we introduce a pre-integration factor that combines ego-velocity and gyroscope data. Additionally, leveraging the stable RCS characteristics of radar, we design a corresponding point selection method based on normal direction and propose a scan-to-submap point cloud registration technique weighted by RCS intensity. Finally, we validate the reliability and localization accuracy of our framework using both our own dataset and the NTU dataset. Experimental results show that the proposed DGRO system outperforms traditional 4D radar odometry methods, especially in environments with slow speeds and fewer dynamic objects.

Pulsar Signal Adaptive Surrogate Modeling

As the number of spacecraft heading beyond Earth’s orbit increased in recent years, autonomous navigation solutions have become increasingly important. One such solution is pulsar-based navigation. The availability of pulsar signals for simulations and HIL testing is essential for the development of pulsar-based navigation. This study proposes a method to develop a surrogate model of pulsar signals based on radio pulsar observations. The selection of suitable pulsars for the radio telescope is discussed, and a series of observations are conducted. The collected data are processed using the PRESTO software, and the pulsar parameters for the model are derived. Unlike current pulsar signal models, the proposed model anticipates pulsar signal parameters to change over time. It can provide dynamic input parameters for known synthetic pulsar signal generators, resulting in a more realistic signal.

Path Tracing-Inspired Modeling of Non-Line-of-Sight SPAD Data.

Non-Line of Sight (NLOS) imaging has gained attention for its ability to detect and reconstruct objects beyond the direct line of sight, using scattered light, with applications in surveillance and autonomous navigation. This paper presents a versatile framework for modeling the temporal distribution of photon detections in direct Time of Flight (dToF) Lidar NLOS systems. Our approach accurately accounts for key factors such as material reflectivity, object distance, and occlusion by utilizing a proof-of-principle simulation realized with the Unreal Engine. By generating likelihood distributions for photon detections over time, we propose a mechanism for the simulation of NLOS imaging data, facilitating the optimization of NLOS systems and the development of novel reconstruction algorithms. The framework allows for the analysis of individual components of photon return distributions, yielding results consistent with prior experimental data and providing insights into the effects of extended surfaces and multi-path scattering. We introduce an optimized secondary scattering approach that captures critical multi-path information with reduced computational cost. This work provides a robust tool for the design and improvement of dToF SPAD Lidar-based NLOS imaging systems.

Advancing Autonomous Vehicle Navigation through Hybrid Fuzzy-Neural Network Training Systems

In the evolving realm of autonomous vehicle navigation, the integration of fuzzy logic and neural networks presents a formidable challenge, particularly in the context of real-time, on-the-fly neural network training. This paper addresses the gap in dynamic and adaptable training methods necessary for navigating unpredictable environments with limited computational resources. The primary objective of our study is to empirically validate a hybrid training approach that combines fuzzy logic with back-propagation learning algorithms, aiming to optimize neural network performance under hardware constraints. Our methodology leverages a fuzzy logic trainer to provide initial training sets dynamically, which guide the neural network in adjusting its weights in real time, thus facilitating adaptive learning during navigation tasks. The findings reveal that this integrated approach not only enhances the learning efficiency of neural networks but also significantly improves navigation accuracy in real-time scenarios. These advancements contribute to the field by demonstrating the feasibility of deploying more adaptable and robust autonomous navigation systems, potentially expanding their application in more diverse and challenging environments.

A Vision-based Robotic Navigation Method Using an Evolutionary and Fuzzy Q-Learning Approach

The paper presents a Fuzzy Q-Learning (FQL) and optical flow based autonomous navigation approach. The FQL method takes decisions in an unknown environment and without mapping, using motion information and through a reinforcement signal into an evolutionary algorithm. The reinforcement signal is calculated by estimating the optical flow densities in areas of the camera to determine whether they are “dense” or “thin” which has a relationship with the proximity of objects. The results obtained show that the present approach improves the rate of learning compared with a method with a simple reward system and without the evolutionary component. The proposed system was implemented in a virtual robotics system using the CoppeliaSim software and in communication with Python.

Understanding amorphous gorge scenes based on the projection of spatial textures

Geographic information and topographical understanding are crucial for effective field environmental monitoring and autonomous navigation. Unfortunately, implementing this knowledge is often difficult due to unsolved problems such as understanding disorganized and unstructured mountainous regions. Traditional three-dimensional (3D) scene understanding methods from 3D point clouds or fused data are energy-consuming and unsuitable for a resource-constrained unmanned aerial vehicle (UAV) with limited computation, memory, and energy. This study proposes a methodology to understand unstructured gorges and reconstruct them in 3D environments from a low-cost monocular camera. The new approach assigns mountain texture projections to clusters and reshapes surfaces of unstructured mountains to follow geometric constraints regarding position and orientation. The relative relationships between surfaces and vanishing points are then analyzed to understand and reconstruct amorphous gorge scenes. This method is practical and efficient for navigating amorphous gorges based on understanding unstructured mountains. Fortunately, this method requires no prior training and is robust to color and illumination. We compared the percentage of incorrectly classified pixels with the ground truth. Experimental results have demonstrated that our approach can successfully understand amorphous gorges, meeting the requirements for autonomous flying in mountainous environments.

Autonomous navigation method for planetary landing based on observability analysis of sequential images

Autonomous navigation method for planetary landing based on observability analysis of sequential images

Evaluating an autonomous electric robot for real farming applications

Mobile robotic technologies are emerging in agriculture in recent years, driven by labor shortages, the need of precision farm management, and the improvement of farmers' quality of life. Mobile robotic technologies, both aerial and ground, have been considered as one of the most important and promising solutions to address agricultural challenges. Nevertheless, the implementation rate in farms is still low. The aim of this study was to evaluate the performance of a mobile agricultural ground robot to assess its possible implementation in the farm environment. The mobility performances as well as the energy consumption of the mobile robot were evaluated considering different interaction systems (teleoperated, autonomous driving) and application areas (weeding, tilling). The results show that the forward speed of the tested mobile robot is affected by the towed load, up to 230 kg, and that the robot could be driven continuously for 2 h in the field without decreasing its performances at a maximum speed between 0.71–0.77 m s-1. The autonomy of the tested robot is limited, but this is mainly due to the battery technology used. The mobile robot effectively tow implements for weeding and tilling operations managing grass with a single pass (40 % weed removal) and improving soil bulk density. The average energy consumption across the different settings was about 1.43 kWh for each hour of use. Moreover, although the RTK-GNSS autonomous navigation used was a low-cost system, the results showed a good accuracy, which allowed the mobile robot to navigate autonomously in the 2 m inter-row of the vineyard. Finally, the working capacity of the mobile robots, considering the implements used, is on average 0.29 ha h-1. This study provides further evidence for the possible use of ground mobile robot in agriculture to carry out autonomous or semi-autonomous operations.

Evaluation of Autonomous Navigation and Path Accuracy for Unmanned Surface Vehicles Using IMU Sensor and GPS Data

The development of autonomous systems for environmental monitoring is becoming increasingly important. This project focuses on creating a robotic marine vessel, Aqua Sense, for efficient water quality monitoring. Existing methods are often labour-intensive, time-consuming, and lack real-time data capability. The objective is to develop a robotic vessel with autonomous navigation for water quality applications. A comprehensive design approach, including a literature review and careful component selection, was employed. Key components include the Arduino Mega, Blue Robotics T200 thruster, GlobalSat BU-353N5 GPS sensor, BNO055 Inertial Measurement Unit (IMU), and a Li-po 11.1v battery. In this paper, several field trials data collection is presented to evaluate the navigation algorithm. These findings validate the feasibility of Aqua Sense for real-world applications. The varying distance errors at points B and C demonstrate an error range, with a maximum error of approximately 1.130m at point B (about 0.11% of the target location) and a significant variation at point C, reaching up to 1.660m (approximately 0.15%).

Convolutional-Neural-Network-Based Autonomous Navigation of Hera Mission Around Didymos

The European Space Agency (ESA)’s Hera mission requires autonomous visual-based navigation in order to safely orbit around the target binary asteroid system Didymos and its moon Dimorphos in 2027. Nevertheless, the performance of optical-based navigation systems depends on the intrinsic properties of the image, such as high Sun phase angles, the presence of other bodies, and, especially, the irregular shape of the target. Therefore, to improve the navigation performance, thermal and/or range measurements from additional onboard instruments are usually needed to complement optical measurements. However, this work addresses these challenges by developing a fully visual-based autonomous navigation system using a convolutional-neural-network (CNN)-based image processing (IP) algorithm and applying it to the detailed characterization phase of the proximity operation of the mission. The images taken by the onboard camera are processed by the CNN-based IP algorithm that estimates the position of the geometrical centers of Didymos and Dimorphos, the range from Didymos, and the associated covariances. The results show that the developed algorithm can be used for a fully visual-based navigation for the position of the Hera spacecraft around the target with good robustness and accuracy.

GreenHopper: The Danish spearhead towards autonomous waterborne mobility

Abstract GreenHopper is the first Danish zero-emission ferry developed as a test platform for autonomous waterborne navigation technologies. The paper presents technology development within the innovation project ShippingLab Autonomy, which led to the commissioning of GreenHopper at Limfjorden (DK) in December 2022. The technology research resulted in a holistic system architecture for surface vessel autonomy, based on distribution of functionality and responsibility on software modules, similar to the structure observed in the International Maritime Organization (IMO) Seafarers Training Certification and Watch-keeping (STCW) regulatory framework. The paper shows how this approach results in an architecture that supports safe behaviours of individual modules and of autonomous navigation at a system level. The paper presents the individual modules, specific features and benefits. Elements of the regulatory framework are highlighted to poise technology approval by maritime authorities. The paper reflects on lessons learned, discusses continued technology validation in dedicated operational scenarios.

Tour of Asteroids for Characterization Observations (TACO): A Planetary Defense Asteroid Tour Concept

Asteroid impacts potentially represent a substantial threat to humanity, but one that we can plan for and mitigate. To design an effective asteroid mitigation mission, however, it is important to have as detailed knowledge of the asteroid threat as possible. Our understanding of a newly discovered object will generally derive from our understanding of the near-Earth object population, and in cases where there is no time for a reconnaissance mission prior to deflection or disruption, we may need to lean heavily on any existing data of similar objects. The Tour of Asteroids for Characterization Observations (TACO) mission concept would fill key gaps in the characterization knowledge needed to plan an effective response to an asteroid threat. A tour targeting potentially hazardous asteroids and focused on reconnaissance objectives specifically relevant for planetary defense would also test instruments and technologies (e.g., autonomous navigation, high-rate gimbals) ahead of when they are actually required in response to a threat. Testing these capabilities is identified as a need in the National Near-Earth Object Preparedness Strategy and Action Plan. The TACO tour concept is specifically designed to measure the most important asteroid properties for planetary defense, including mass, size/shape, surface and near-surface structure, presence of satellites, and composition. These measurements can be obtained using a nominal payload, including a narrow-angle camera, a thermal infrared imager, and deployed test masses for gravity science.

Counterfactual-based two-stage trajectory prediction framework for autonomous vehicle navigation systems

Abstract Accurate prediction of a vehicle’s trajectory is essential for autonomous driving and robotic operations in a complex, dynamic environment. However, current methods that rely on historical and environmental cues to predict future trajectories make the inner distance between testing and training environments often overlooked. To address this issue, we propose a counterfactual analysis method specifically designed for vehicle trajectory prediction. We begin by constructing a causal graph that includes historical trajectories, future trajectories, and environmental interactions. This graph captures the causal relationships between these elements. Our approach directly intervenes on the trajectory itself, effectively cutting off the inference from the environment to the trajectory. By doing so, we mitigate the negative impact of environmental bias. Furthermore, by comparing observed and intervened trajectory clues, we highlight the influence of environmental distance. Our goal is to improve the accuracy of trajectory prediction. Our counterfactual analysis module integrates seamlessly into various baseline prediction models. Experimental data demonstrates that our method outperforms others on publicly available Argoverse2 motion forecasting benchmarks.

Manoeuvring in shallow and confined water with model predictive track controllers

Abstract Autonomous and computer-assisted navigation, for example through portable pilot units and data-driven decision supportive systems, is in full development. Remote-controlled vessels (manned but without a captain steering continuously on board) are used in Belgium (by Seafar in inland shipping) but also in other countries. A model predictive controller has been used to investigate environmental and operational parameters for the passage of the river Seine in Paris by different ship types. These fast-time simulations have been compared with simulations in real time executed by skippers on a ship manoeuvring simulator or measured real life tracks. A good predictability of the real situation requires accurate manoeuvring models. But what if this accuracy cannot be guaranteed? An illustration is made by comparing the tracks if, in the prescience prediction phase for the controller, wind or bank effects are neglected or doubled. The development of model predictive deep reinforcement learning control algorithms for navigation will be one of the challenges of the Data-driven Smart Shipping project that started May 2024.