Unstructured Outdoor Environments Research Articles

Enhancing computer image recognition with improved image algorithms

Advances in computer image recognition have significantly impacted many industries, including healthcare, security and autonomous systems. This paper aims to explore the potential of improving image algorithms to enhance computer image recognition. Specifically, we will focus on regression methods as a means to improve the accuracy and efficiency of identifying images. In this study, we will analyze various regression techniques and their applications in computer image recognition, as well as the resulting performance improvements through detailed examples and data analysis. This paper deals with the problems related to visual image processing in outdoor unstructured environment. Finally, the heterogeneous patterns are converted into the same pattern, and the heterogeneous patterns are extracted from the fusion features of data modes. The simulation results show that the perception ability and recognition ability of outdoor image recognition in complex environment are improved.

Snail-inspired robotic swarms: a hybrid connector drives collective adaptation in unstructured outdoor environments

Terrestrial self-reconfigurable robot swarms offer adaptable solutions for various tasks. However, most existing swarms are limited to controlled indoor settings, and often compromise stability due to their freeform connections. To address these issues, we present a snail robotic swarm system inspired by land snails, tailored for unstructured environments. Our system also employs a two-mode connection mechanism, drawing from the adhesive capabilities of land snails. The free mode, mirroring a snail’s natural locomotion, leverages magnet-embedded tracks for freeform mobility, thereby enhancing adaptability and efficiency. The strong mode, analogous to a snail’s response to disturbance, employs a vacuum sucker with directional polymer stalks for robust adhesion. By assigning specific functions to each mode, our system achieves a balance between mobility and secure connections. Outdoor experiments demonstrate the capabilities of individual robots and the exceptional synergy within the swarm. This research advances the real-world applications of terrestrial robotic swarms in unstructured environments.

Correction to “Aerial online mapping on‐board system by real‐time object detection for UGV path generation in unstructured outdoor environments”

Correction to “Aerial online mapping on‐board system by real‐time object detection for UGV path generation in unstructured outdoor environments”

Kernel-based diffusion approximated Markov decision processes for autonomous navigation and control on unstructured terrains

We propose a diffusion approximation method to the continuous-state Markov decision processes that can be utilized to address autonomous navigation and control in unstructured off-road environments. In contrast to most decision-theoretic planning frameworks that assume fully known state transition models, we design a method that eliminates such a strong assumption that is often extremely difficult to engineer in reality. We first take the second-order Taylor expansion of the value function. The Bellman optimality equation is then approximated by a partial differential equation, which only relies on the first and second moments of the transition model. By combining the kernel representation of the value function, we design an efficient policy iteration algorithm whose policy evaluation step can be represented as a linear system of equations characterized by a finite set of supporting states. We first validate the proposed method through extensive simulations in 2D obstacle avoidance and 2.5D terrain navigation problems. The results show that the proposed approach leads to a much superior performance over several baselines. We then develop a system that integrates our decision-making framework with onboard perception and conduct real-world experiments in both cluttered indoor and unstructured outdoor environments. The results from the physical systems further demonstrate the applicability of our method in challenging real-world environments.

Challenges and Solutions for Autonomous Ground Robot Scene Understanding and Navigation in Unstructured Outdoor Environments: A Review

The capabilities of autonomous mobile robotic systems have been steadily improving due to recent advancements in computer science, engineering, and related disciplines such as cognitive science. In controlled environments, robots have achieved relatively high levels of autonomy. In more unstructured environments, however, the development of fully autonomous mobile robots remains challenging due to the complexity of understanding these environments. Many autonomous mobile robots use classical, learning-based or hybrid approaches for navigation. More recent learning-based methods may replace the complete navigation pipeline or selected stages of the classical approach. For effective deployment, autonomous robots must understand their external environments at a sophisticated level according to their intended applications. Therefore, in addition to robot perception, scene analysis and higher-level scene understanding (e.g., traversable/non-traversable, rough or smooth terrain, etc.) are required for autonomous robot navigation in unstructured outdoor environments. This paper provides a comprehensive review and critical analysis of these methods in the context of their applications to the problems of robot perception and scene understanding in unstructured environments and the related problems of localisation, environment mapping and path planning. State-of-the-art sensor fusion methods and multimodal scene understanding approaches are also discussed and evaluated within this context. The paper concludes with an in-depth discussion regarding the current state of the autonomous ground robot navigation challenge in unstructured outdoor environments and the most promising future research directions to overcome these challenges.

Aerial online mapping on‐board system by real‐time object detection for UGV path generation in unstructured outdoor environments

AbstractAn optimal path provides efficient operation of unmanned ground vehicles (UGVs) for many kinds of tasks such as transportation, exploration, surveillance, and search and rescue in unstructured areas that include various unexpected obstacles. Various onboard sensors such as LiDAR, radar, sonar, and cameras are used to detect obstacles around the UGVs. However, their range of view is often limited by movable obstacles or barriers, resulting in inefficient path generation. Here, we present the aerial online mapping system to generate an efficient path for a UGV on a two‐dimensional map. The map is updated by projecting obstacles detected in the aerial images taken by an unmanned aerial vehicle through an object detector based on a conventional convolutional neural network. The proposed system is implemented in real‐time by a skid steering ground vehicle and a quadcopter with relatively small, low‐cost embedded systems. The frameworks and each module of the systems are given in detail to evaluate the performance. The system is also demonstrated in unstructured outdoor environments such as in a football field and a park with unreliable communication links. The results show that the aerial online mapping is effective in path generation for autonomous UGVs in real environments.

Where am I heading? A robust approach for orientation estimation of autonomous agricultural robots

Reliable knowledge of the vehicle heading plays a significant role in the autonomous navigation of agricultural Unmanned Ground Vehicles (UGVs), especially in the context of unstructured outdoor environments such as rural and forestry scenarios. However, achieving this information with an acceptable degree of confidence is a non-trivial task and still an open field of research. Expensive solutions are available on the market, but they often discourage most farmers due to the large investments needed for the startup. This paper introduces a novel algorithmic solution for reliable evaluation of the absolute vehicle heading, grounded on adaptive Kalman filtering with input evaluation via linear regression analysis. The proposed approach provides a functional and affordable solution to the heading estimation problem that can be used in real-world applications. The system is validated through an extensive experimental campaign using an all-terrain tracked rover operating in agricultural settings, showing good accuracy compared to other approaches, such as a dual GPS method found in the literature.

Analysis of State-of-the-art Software Solutions for Unstructured Outdoor Environment Perception in Goal-oriented Navigation of Off-road Unmanned Ground Vehicles

This paper presents a comprehensive analysis of state-of-the-art software solutions designed for the perception of unstructured outdoor environments, with a focus on their applicability in goal-oriented navigation for off-road unmanned ground vehicles (UGVs). The analysis encompasses the evaluation of various sensors for UGV navigation, comparison of methods for extracting perceptual information from sensor data, and the development of a perception system based on the findings. The proposed system leverages advancements in simultaneous localization and mapping (SLAM), visual odometry, and traversability segmentation using 3D LiDAR and visual systems. The analysis suggests a phased approach for the development of autonomous navigation, where the initial phase relies on complex SLAM and teleoperation assistance, and the subsequent phase introduces an autonomous mode utilizing semantical terrain representation.

A Study of Real-time Semantic Segmentation Performance Improvement in Unstructured Outdoor Environment

Semantic segmentation in autonomous driving for unstructured environments is challenging due to the presence of uneven terrains, unstructured class boundaries, irregular features and strong textures. Current off-road datasets exhibit difficulties like class imbalance and understanding of varying environmental topography. To overcome these issues, we propose a deep learning framework for semantic segmentation that involves a pooled class semantic segmentation with five classes. The evaluation of the framework is carried out on two off-road driving datasets, RUGD and TAS500. The results show that our proposed method achieves high accuracy and real-time performance.

WayFAST: Navigation With Predictive Traversability in the Field

We present a self-supervised approach for learning to predict traversable paths for wheeled mobile robots that require good traction to navigate. Our algorithm, termed WayFAST (Waypoint Free Autonomous Systems for Traversability), uses RGB and depth data, along with navigation experience, to autonomously generate traversable paths in outdoor unstructured environments. Our key inspiration is that traction can be estimated for rolling robots using kinodynamic models. Using traction estimates provided by an online receding horizon estimator, we are able to train a traversability prediction neural network in a self-supervised manner, without requiring heuristics utilized by previous methods. We demonstrate the effectiveness of WayFAST through extensive field testing in varying environments, ranging from sandy dry beaches to forest canopies and snow covered grass fields. Our results clearly demonstrate that WayFAST can learn to avoid geometric obstacles as well as untraversable terrain, such as snow, which would be difficult to avoid with sensors that provide only geometric data, such as LiDAR. Furthermore, we show that our training pipeline based on online traction estimates is more data-efficient than other heuristic-based methods.

GA-Nav: Efficient Terrain Segmentation for Robot Navigation in Unstructured Outdoor Environments

We propose GA-Nav, a novel group-wise attention mechanism to identify safe and navigable regions in unstructured environments from RGB images. Our group-wise attention method extracts multi-scale features from each type of terrain independently and classifies terrains based on their navigability levels using coarse-grained semantic segmentation. Our novel loss can be embedded within any backbone network to explicitly focus on the different groups’ features, at a low spatial resolution. Our design leads to efficient inference while maintaining a high level of accuracy compared to existing SOTA methods. Our extensive evaluations on the RUGD and RELLIS-3D datasets shows that GA-Nav achieves the state-of-the-art performance on RUGD and RELLIS-3D datasets. We interface GA-Nav with a deep reinforcement learning-based navigation algorithm and highlight its benefits in terms of navigation in real-world unstructured terrains. We integrate our GA-Nav-based navigation algorithm with ClearPath Jackal and Husky robots, and observe an improvement in terms of navigation success rate and better trajectory selections.

Efficient and Probabilistic Adaptive Voxel Mapping for Accurate Online LiDAR Odometry

This letter proposes an efficient and probabilistic adaptive voxel mapping method for LiDAR odometry. The map is a collection of voxels; each contains one plane feature that enables the probabilistic representation of the environment and accurate registration of a new LiDAR scan. We further analyze the need for coarse-to-fine voxel mapping and then use a novel voxel map organized by a Hash table and octrees to build and update the map efficiently. We apply the proposed voxel map to an iterated extended Kalman filter and construct a maximum a posteriori probability problem for pose estimation. Experiments on the open KITTI dataset show the high accuracy and efficiency of our method compared to other state-of-the-art methods. Experiments on indoor and unstructured outdoor environments with solid-state LiDAR and non-repetitive scanning LiDAR further verify the adaptability of our mapping method to different environments and LiDAR scanning patterns (see our attached video <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> ). Our codes and dataset are open-sourced on Github <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>

Keeping an Eye on Things: Deep Learned Features for Long-Term Visual Localization

In this letter, we learn visual features that we use to first build a map and then localize a robot driving autonomously across a full day of lighting change, including in the dark. We train a neural network to predict sparse keypoints with associated descriptors and scores that can be used together with a classical pose estimator for localization. Our training pipeline includes a differentiable pose estimator such that training can be supervised with ground truth poses from data collected earlier, in our case from 2016 and 2017 gathered with multi-experience Visual Teach and Repeat (VT&R). We insert the learned features into the existing VT&R pipeline to perform closed-loop path following in unstructured outdoor environments. We show successful path following across all lighting conditions despite the robot’s map being constructed using daylight conditions. Moreover, we explore generalizability of the features by driving the robot across all lighting conditions in new areas not present in the feature training dataset. In all, we validated our approach with 35.5 km of autonomous path following experiments in challenging conditions.

OMNI-DRL: Learning to Fly in Forests with Omnidirectional Images

Perception is crucial for drone obstacle avoidance in complex, static, and unstructured outdoor environments. However, most navigation solutions based on Deep Reinforcement Learning (DRL) use limited Field-Of-View (FOV) images as input. In this paper, we demonstrate that omnidirectional images improve these methods. Thus, we provide a comparative benchmark of several visual modalities for navigation: ground truth depth, ground truth semantic segmentation, and RGB images. These exhaustive comparisons reveal that it is superior to use an omnidirectional camera to navigate with classical DRL methods. Finally, we show in two different virtual forest environments that adapting the convolution to take into account spherical distortions improves the results even more.

A real-time low-computation cost human-following framework in outdoor environment for legged robots

Legged robots have potential advantages in mobility compared with wheeled robots. Hence, legged robots are widely utilized in outdoor unstructured environments. Human-following operation is one of important tasks for outdoor robots. However, most current human-following strategies requires large amount of computation resource hence difficult to be applied to legged robots. This paper proposes real-time low-computation cost human-following framework in outdoor environment for legged robots. Our method takes a full consideration of the differences between legged robots and wheeled robots. Firstly, an on-line extrinsic calibration method is proposed to calculate the camera coordinate and the world coordinate system. Then, a real-time low-computation cost human-following method utilizing RGBD cameras and 3D LIDAR is proposed. The robot motion considers tracking the leading person while avoiding obstacles. Furthermore, a dynamic alternating tripod trotting gait is developed to control the robot to follow the leading person. Finally, the method is implemented and tested on a hexapod robot Qingzhui with indoor and outdoor experiments. The framework proposed in this paper can be a valuable reference for other legged robots when operated in outdoor environments.

Obstacle Detection System for Agricultural Mobile Robot Application Using RGB-D Cameras.

Mobile robots designed for agricultural tasks need to deal with challenging outdoor unstructured environments that usually have dynamic and static obstacles. This assumption significantly limits the number of mapping, path planning, and navigation algorithms to be used in this application. As a representative case, the autonomous lawn mowing robot considered in this work is required to determine the working area and to detect obstacles simultaneously, which is a key feature for its working efficiency and safety. In this context, RGB-D cameras are the optimal solution, providing a scene image including depth data with a compromise between precision and sensor cost. For this reason, the obstacle detection effectiveness and precision depend significantly on the sensors used, and the information processing approach has an impact on the avoidance performance. The study presented in this work aims to determine the obstacle mapping accuracy considering both hardware- and information processing-related uncertainties. The proposed evaluation is based on artificial and real data to compute the accuracy-related performance metrics. The results show that the proposed image and depth data processing pipeline introduces an additional distortion of 38 cm.

Mapping in unstructured natural environment: a sensor fusion framework for wearable sensor suites

We present a generalized mapping framework that can withstand the challenges incurred by working in unstructured outdoor environments, such as a snowy forest. The proposed method takes advantage of a sensor fusion scheme, where sensors such as cameras and lidars are used in order to reconstruct the surrounding natural environment. Although mapping techniques such as SLAM and ICP cannot themselves properly handle the complexity of natural scenes, they do have the potential to contribute to the global solution in a proposed sensor fusion scheme, based on a factor graph architecture. In this paper, we propose an innovative map registration scheme for visual maps, and show how it can improve the reconstruction quality after data fusion. We also analyze the behavior and sensitivity of factor graphs to uncertainties, by comparing the residual error with different parameter combinations such as variances, using an exhaustive grid search with ground truth comparison. Finally, we suggest an ICP-inferred loop closure, capable of compensating position and attitude drift. The experiments are carried out by recording in a snowy forest using a wearable sensor suite. In the experiments, ground truth was acquired using a millimeter-accurate total station. The proposed framework is shown to be robust and likewise capable of providing estimates that are otherwise unattainable using classic techniques, such as visual SLAM and ICP for lasers. Finally, a visible improvement in the map reconstruction quality is shown, and the proposed framework achieves a translation error of 0.36 m.

Multi-sensor fusion approach based on nonlinear H∞ filter with interval type 2 fuzzy adaptive parameters tuning for unmanned vehicle localization

In most applications of autonomous navigation, the state of a system must be estimated from noisy sensors. Accurate estimation of the true system state can be achieved using data fusion algorithms. Furthermore, the fusion scheme can be affected by many factors such as modeling errors and parameters uncertainties. The gaps and inconsistencies due to the sensors noise and modeling errors can be reached with robust nonlinear filtering. In this article, a new framework has been developed for data fusion algorithms based on nonlinear NH∞ filter with fuzzy adaptive bound and adaptive disturbances attenuations. Type-1 Fuzzy Adaptive NH∞ algorithm has been proposed and compared with the Interval Type 2 Fuzzy Adaptive NH∞, for unmanned vehicle localization. The proposed algorithms fuse data from low-cost sensors using inertial navigation system, Global Positioning System and monocular vision. Type-1 Fuzzy Adaptive NH∞ and Interval Type 2 Fuzzy Adaptive NH∞ algorithms, adaptively, handle the effects of noisy sensors, parameters uncertainties and modeling errors. Both algorithms use adaptive bounds [Formula: see text] and adaptive disturbance attenuation [Formula: see text] for higher-order terms of the Taylor development. The adaptive bounds consider issues associated to Gyro drift, image distortion and Global Positioning System dilution of precision which make the proposed algorithms more accurate than the classical NH∞. The validating experiment and the efficiency of Type-1 Fuzzy Adaptive NH∞ and Interval Type 2 Fuzzy Adaptive NH∞ have been conducted in real scenario and in unstructured outdoor environment without any a priori knowledge of the evolution of the vehicle. The experimental results have demonstrated the robustness and efficiency of these two filtering strategies against uncertainties and their online sensor switching capability. The Interval Type 2 Fuzzy Adaptive NH∞ algorithm provides more accuracy and robustness compared to the Type-1 Fuzzy Adaptive NH∞.

基于道路边缘运动障碍物检测算法的农业导航机器人道路识别技术

In order to recognize the road effectively, agricultural robots mainly rely on the tracking and detection data of road obstacles. Traditional obstacle detection mainly studies how to use multiple fusion methods such as vision and laser to analyse structured and simplified indoor scenes. The working environment of agricultural robots is a typical unstructured outdoor environment. Therefore, based on the environmental characteristics of agricultural robot navigation, the mean displacement algorithm is introduced to detect and study the obstacles aiming at the road edge. After explaining the advantages and principle flow of the mean displacement algorithm to effectively realize motion capture, the feasibility of target location and tracking research is discussed. After that, the bottom data acquisition and analysis model is constructed based on the road navigation data of agricultural robots. To capture the movement obstacles of road edge and build the foundation of road recognition technology. In order to improve the effectiveness of motion obstacle capture and detection, a moving target detection algorithm is proposed to optimize and update the mean displacement algorithm, and constructs a feature-oriented hybrid algorithm motion capture model. The simulation results indicate that the proposed optimization model can effectively improve the tracking efficiency of non-rigid targets in outdoor environment, and the number of evaluation iterations can reach 3.5621 times per frame, which shows that the research has good theoretical and practical value.

3D Registration and Integrated Segmentation Framework for Heterogeneous Unmanned Robotic Systems

The paper proposes a novel framework for registering and segmenting 3D point clouds of large-scale natural terrain and complex environments coming from a multisensor heterogeneous robotics system, consisting of unmanned aerial and ground vehicles. This framework involves data acquisition and pre-processing, 3D heterogeneous registration and integrated multi-sensor based segmentation modules. The first module provides robust and accurate homogeneous registrations of 3D environmental models based on sensors’ measurements acquired from the ground (UGV) and aerial (UAV) robots. For 3D UGV registration, we proposed a novel local minima escape ICP (LME-ICP) method, which is based on the well known iterative closest point (ICP) algorithm extending it by the introduction of our local minima estimation and local minima escape mechanisms. It did not require any prior known pose estimation information acquired from sensing systems like odometry, global positioning system (GPS), or inertial measurement units (IMU). The 3D UAV registration has been performed using the Structure from Motion (SfM) approach. In order to improve and speed up the process of outliers removal for large-scale outdoor environments, we introduced the Fast Cluster Statistical Outlier Removal (FCSOR) method. This method was used to filter out the noise and to downsample the input data, which will spare computational and memory resources for further processing steps. Then, we co-registered a point cloud acquired from a laser ranger (UGV) and a point cloud generated from images (UAV) generated by the SfM method. The 3D heterogeneous module consists of a semi-automated 3D scan registration system, developed with the aim to overcome the shortcomings of the existing fully automated 3D registration approaches. This semi-automated registration system is based on the novel Scale Invariant Registration Method (SIRM). The SIRM provides the initial scaling between two heterogenous point clouds and provides an adaptive mechanism for tuning the mean scale, based on the difference between two consecutive estimated point clouds’ alignment error values. Once aligned, the resulting homogeneous ground-aerial point cloud is further processed by a segmentation module. For this purpose, we have proposed a system for integrated multi-sensor based segmentation of 3D point clouds. This system followed a two steps sequence: ground-object segmentation and color-based region-growing segmentation. The experimental validation of the proposed 3D heterogeneous registration and integrated segmentation framework was performed on large-scale datasets representing unstructured outdoor environments, demonstrating the potential and benefits of the proposed semi-automated 3D registration system in real-world environments.