Unstructured Outdoor Environments Research Articles

Multi-Task Regression-Based Learning for Autonomous Unmanned Aerial Vehicle Flight Control Within Unstructured Outdoor Environments

Increased growth in the global Unmanned Aerial Vehicles (UAV) (drone) industry has expanded possibilities for fully autonomous UAV applications. A particular application which has in part motivated this research is the use of UAV in wide area search and surveillance operations in unstructured outdoor environments. The critical issue with such environments is the lack of structured features that could aid in autonomous flight, such as road lines or paths. In this paper, we propose an End-to-End Multi-Task Regression-based Learning approach capable of defining flight commands for navigation and exploration under the forest canopy, regardless of the presence of trails or additional sensors (i.e. GPS). Training and testing are performed using a software in the loop pipeline which allows for a detailed evaluation against state-of-the-art pose estimation techniques. Our extensive experiments demonstrate that our approach excels in performing dense exploration within the required search perimeter, is capable of covering wider search regions, generalises to previously unseen and unexplored environments and outperforms contemporary state-of-the-art techniques.

3D mapping of outdoor environments by scan matching and motion averaging

Building 3D maps of surroundings is a fundamental and important task for autonomous robots operating in large previously unknown semi-structured or unstructured outdoor environments, where external methods for localization like GPS are not always available. Most of existing 3D mapping approaches are based on the local scan matching algorithm, which are difficult to achieve accurate mapping with good efficiency. In this paper, we propose an effective graph-based SLAM approach for 3D mapping of outdoor environments without odometry measurement. It first utilizes scan matching algorithm to sequentially estimate initial robot poses and progressively construct a pose graph, where each vertex denotes the robot pose at a point in time and each edge indicates the constraint between two robot poses. As scan matching algorithm cannot be error-free, the estimation of robot poses suffers from the error accumulation problem. To address this issue, the hypothesis of loop closure can be generated by an adaptive search circle and then verified by the global scan matching algorithm. Once the loop closure is verified, more edges should be added into the pose graph, which is subsequently optimized by the weighted motion averaging algorithm so as to obtain accurate robot poses for 3D mapping. Experimental results carried out on multiple outdoor environment data sets show that the proposed approach can build 3D maps with good performance.

Autonomous Unmanned Aerial Vehicles Filming In Dynamic Unstructured Outdoor Environments [Applications Corner

The recent mass commercialization of affordable unmanned aerial vehicles (UAVs), known as drones, has significantly altered the media production landscape, allowing for the easy acquisition of impressive aerial footage. Relevant applications include the production of movies, TV shows, or commercials as well as the filming of outdoor events or news stories. In the near future, increased drone autonomy is expected to reduce shooting costs and shift focus to the creative process, rather than the minutiae of UAV operation. This article introduces and surveys the emerging field of autonomous UAV filming and familiarizes the reader with the inherent signal processing aspects and challenges.

Open-sector rapid-reactive collision avoidance: Application in aerial robot navigation through outdoor unstructured environments

A new reactive collision avoidance method for navigation of aerial robots (such as unmanned aerial vehicles (UAVs)) in unstructured urban/suburban environments is presented. Small form-factor aerial robots, such as quadcopters, often have limited payload capacity, flight time, processing power, and sensing capabilities. To enhance the capabilities of such vehicles without increasing weight or computing power, a reactive collision avoidance method based on open sectors is described. The method utilizes information from a two-dimensional laser scan of the environment and a short-term memory of past actions and can rapidly circumvent obstacles in outdoor urban/suburban environments. With no map required, the method enables the robot to react quickly and navigate even when the enivornment changes. Furthermore, the low computational requirement of the method allows the robot to quickly react to unknown obstacles that may be poorly represented in the scan, such as trees with branches and leaves. The method is validated in simulation results and through physical experiments on a prototype quadcopter system, where results show the robot flying smoothly around obstacles at a relatively high speed (3 m/s).

Selective memory: Recalling relevant experience for long‐term visual localization

AbstractVisual navigation is a key enabling technology for autonomous mobile vehicles. The ability to provide large‐scale, long‐term navigation using low‐cost, low‐power vision sensors is appealing for industrial applications. A crucial requirement for long‐term navigation systems is the ability to localize in environments whose appearance is constantly changing over time—due to lighting, weather, seasons, and physical changes. This paper presents a multiexperience localization (MEL) system that uses a powerful map representation—storing every visual experience in layers—that does not make assumptions about underlying appearance modalities and generators. Our localization system provides real‐time performance by selecting online, a subset of experiences against which to localize. We achieve this task through a novel experience‐triage algorithm based on collaborative filtering, which selects experiences relevant to the live view, outperforming competing techniques. Based on classical memory‐based recommender systems, this technique also enables landmark‐level recommendations, is entirely online, and requires no training data. We demonstrate the capabilities of the MEL system in the context of long‐term autonomous path following in unstructured outdoor environments with a challenging 100‐day field experiment through day, night, snow, spring, and summer. We furthermore provide offline analysis comparing our system to several state‐of‐the‐art alternatives. We show that the combination of the novel methods presented in this paper enable full use of incredibly rich multiexperience maps, opening the door to robust long‐term visual localization.

Multi-resolution and particle filter based global cooperated localization method

Cooperation & coordination among heterogeneous robots (e.g., UAV & UGV, UAV & USV) have obtained extensive attentions due to its greatly enhanced perception ability, especially when facing complicated and unstructured outdoor environments. For multi-type robots cooperation task in the GPS-denied environment, cooperate localization is one of the most important task with great challenges which combine the aspects of data collection of different type sensors, adaption to the environment feature and perspective variation. In this paper, we proposed a multi-resolution enhanced cooperated localization method for the multi-type robots task. Based on multi-resolution iterative closest point (MRICP) and particle filter (PF), the proposed method could achieve a fast cooperation localization even without a given initial pose estimation. Firstly, a new registration evaluation function is introduced to value the registration efficiency and registration tendency. Secondly, to handle the posterior distribution problem in the PF based global localization, a map updater model is taken to control map scale, resolution level and the particles size to balance the registration efficiency and accuracy. Thirdly, a self-adoption particle resample scheme could deal with the sample impoverishment/degeneracy problem. Experiments with an unmanned aerial vehicle (UAV) and an unmanned surface vehicle (USV) are conducted for cooperate localization test at a water bay, and the results are analyzed to verify the feasibility and validity of the proposed method.

DroNet: Learning to Fly by Driving

Civilian drones are soon expected to be used in a wide variety of tasks, such as aerial surveillance, delivery, or monitoring of existing architectures. Nevertheless, their deployment in urban environments has so far been limited. Indeed, in unstructured and highly dynamic scenarios, drones face numerous challenges to navigate autonomously in a feasible and safe way. In contrast to traditional “map-localize-plan” methods, this letter explores a data-driven approach to cope with the above challenges. To accomplish this, we propose DroNet: a convolutional neural network that can safely drive a drone through the streets of a city. Designed as a fast eight-layers residual network, DroNet produces two outputs for each single input image: A steering angle to keep the drone navigating while avoiding obstacles, and a collision probability to let the UAV recognize dangerous situations and promptly react to them. The challenge is however to collect enough data in an unstructured outdoor environment such as a city. Clearly, having an expert pilot providing training trajectories is not an option given the large amount of data required and, above all, the risk that it involves for other vehicles or pedestrians moving in the streets. Therefore, we propose to train a UAV from data collected by cars and bicycles, which, already integrated into the urban environment, would not endanger other vehicles and pedestrians. Although trained on city streets from the viewpoint of urban vehicles, the navigation policy learned by DroNet is highly generalizable. Indeed, it allows a UAV to successfully fly at relative high altitudes and even in indoor environments, such as parking lots and corridors. To share our findings with the robotics community, we publicly release all our datasets, code, and trained networks.

3D Reconstruction from a Single Still Image Based on Monocular Vision of an Uncalibrated Camera

we propose a framework of combining Machine Learning with Dynamic Optimization for reconstructing scene in 3D automatically from a single still image of unstructured outdoor environment based on monocular vision of an uncalibrated camera. After segmenting image first time, a kind of searching tree strategy based on Bayes rule is used to identify the hierarchy of all areas on occlusion. After superpixel segmenting image second time, the AdaBoost algorithm is applied in the integration detection to the depth of lighting, texture and material. Finally, all the factors above are optimized with constrained conditions, acquiring the whole depthmap of an image. Integrate the source image with its depthmap in point-cloud or bilinear interpolation styles, realizing 3D reconstruction. Experiment in comparisons with typical methods in associated database demonstrates our method improves the reasonability of estimation to the overall 3D architecture of image’s scene to a certain extent. And it does not need any manual assist and any camera model information.

Delayed Monocular SLAM Approach Applied to Unmanned Aerial Vehicles.

In recent years, many researchers have addressed the issue of making Unmanned Aerial Vehicles (UAVs) more and more autonomous. In this context, the state estimation of the vehicle position is a fundamental necessity for any application involving autonomy. However, the problem of position estimation could not be solved in some scenarios, even when a GPS signal is available, for instance, an application requiring performing precision manoeuvres in a complex environment. Therefore, some additional sensory information should be integrated into the system in order to improve accuracy and robustness. In this work, a novel vision-based simultaneous localization and mapping (SLAM) method with application to unmanned aerial vehicles is proposed. One of the contributions of this work is to design and develop a novel technique for estimating features depth which is based on a stochastic technique of triangulation. In the proposed method the camera is mounted over a servo-controlled gimbal that counteracts the changes in attitude of the quadcopter. Due to the above assumption, the overall problem is simplified and it is focused on the position estimation of the aerial vehicle. Also, the tracking process of visual features is made easier due to the stabilized video. Another contribution of this work is to demonstrate that the integration of very noisy GPS measurements into the system for an initial short period of time is enough to initialize the metric scale. The performance of this proposed method is validated by means of experiments with real data carried out in unstructured outdoor environments. A comparative study shows that, when compared with related methods, the proposed approach performs better in terms of accuracy and computational time.

Diagonal recurrent neural networks for parameters identification of terrain based on wheel–soil interaction analysis

Wheeled mobile robots (WMR) are often applied to travel on outdoor unstructured environment, such as loose soil or variable field terrain. Learning the knowledge of terrain has played a significant role for better mobility and stability of WMR. In this study, a diagonal recurrent neural network-based adaptive method is proposed to identify terrain parameters by the platform of a single driving wheel. According to the classical terramechanics model of wheel---soil interaction, a decoupling simplification model is developed by closed-form analytical equations. Five unknown terrain parameters are divided into two groups and included in two complex nonlinear equations. These parameters are used to compute the model outputs of force and torque of wheel---soil interaction. Dynamic back propagation algorithm is applied to update these parameters for compensating the errors between the prediction of neural network and measurable data in real time. The results of simulation show that the terrain parameters can be obtained and approximate the experimental value of terrain parameters when the predictive errors converge to zero.

Development of a Laser-Range-Finder-Based Human Tracking and Control Algorithm for a Marathoner Service Robot

This paper presents a human detection algorithm and an obstacle avoidance algorithm for a marathoner service robot (MSR) that provides a service to a marathoner while training. To be used as a MSR, the mobile robot should have the abilities to follow a running human and avoid dynamically moving obstacles in an unstructured outdoor environment. To detect a human by a laser range finder (LRF), we defined features of the human body in LRF data and employed a support vector data description method. In order to avoid moving obstacles while tracking a running person, we defined a weighted radius for each obstacle using the relative velocity between the robot and an obstacle. For smoothly bypassing obstacles without collision, a dynamic obstacle avoidance algorithm for the MSR is implemented, which directly employed a real-time position vector between the robot and the shortest path around the obstacle. We verified the feasibility of these proposed algorithms through experimentation in different outdoor environments.

A Situationally Aware Voice‐commandable Robotic Forklift Working Alongside People in Unstructured Outdoor Environments

One long‐standing challenge in robotics is the realization of mobile autonomous robots able to operate safely in human workplaces, and be accepted by the human occupants. We describe the development of a multiton robotic forklift intended to operate alongside people and vehicles, handling palletized materials within existing, active outdoor storage facilities. The system has four novel characteristics. The first is a multimodal interface that allows users to efficiently convey task‐level commands to the robot using a combination of pen‐based gestures and natural language speech. These tasks include the manipulation, transport, and placement of palletized cargo within dynamic, human‐occupied warehouses. The second is the robot's ability to learn the visual identity of an object from a single user‐provided example and use the learned model to reliably and persistently detect objects despite significant spatial and temporal excursions. The third is a reliance on local sensing that allows the robot to handle variable palletized cargo and navigate within dynamic, minimally prepared environments without a global positioning system. The fourth concerns the robot's operation in close proximity to people, including its human supervisor, pedestrians who may cross or block its path, moving vehicles, and forklift operators who may climb inside the robot and operate it manually. This is made possible by interaction mechanisms that facilitate safe, effective operation around people. This paper provides a comprehensive description of the system's architecture and implementation, indicating how real‐world operational requirements motivated key design choices. We offer qualitative and quantitative analyses of the robot operating in real settings and discuss the lessons learned from our effort.

Design and comparative evaluation of an iterative contact point estimation method for static stability estimation of mobile actively reconfigurable robots

Due to the advancements of robotic systems, they are able to be employed in more unstructured outdoor environments. In such environments the robot–terrain interaction becomes a highly non-linear function. Several methods were proposed to estimate the robot–terrain interaction: machine learning methods, iterative geometric methods, quasi-static and fully dynamic physics simulations. However, to the best of our knowledge there has been no systematic evaluation comparing those methods.In this paper, we present a newly developed iterative contact point estimation method for static stability estimation of actively reconfigurable robots. This new method is systematically compared to a physics simulation in a comprehensive evaluation. Both interaction models determine the contact points between robot and terrain and facilitate a subsequent static stability prediction. Hence, they can be used in our state space global planner for rough terrain to evaluate the robot’s pose and stability. The analysis also compares deterministic versions of both methods to stochastic versions which account for uncertainty in the robot configuration and the terrain model. The results of this analysis show that the new iterative method is a valid and fast approximate method. It is significantly faster compared to a physics simulation while providing good results in realistic robotic scenarios.

Multi‐sensor‐based online positive learning for drivable region detection

A new method for detecting drivable regions in an unrehearsed and unstructured outdoor environment using multi-sensor information is presented. To achieve this goal, two key methods are developed: (i) robust and effective feature definition using colour and geometry and (ii) online learning algorithm using positive samples for detecting drivable regions. With real data sets, the effect of sensor modality is evaluated and is compared the performance of the algorithm to a cluster-based approach.

Using Embodied Data for Localization and Mapping

Mobile autonomous robots have finally emerged from the confined spaces of structured and controlled indoor environments. To fulfill the promises of ubiquitous robotics in unstructured outdoor environments, robust navigation is a key requirement. The research in the simultaneous localization and mapping (SLAM) community has largely focused on optical sensors to solve this problem, and the fact that the robot is a physical entity has largely been ignored. In this paper, a hierarchical SLAM framework is proposed that takes the interaction of the robot with the environment into account. A sequential Monte Carlo filter is used to generate local map segments with a combination of visual and embodied data associations. Constraints between segments are used to generate globally consistent maps with a focus on suitability for navigation tasks. The proposed method is experimentally verified on two different outdoor robots. The results show that the approach is viable and that the rich modeling of the robot with its environment provides a new modality with the potential for improving existing visual methods and extending the availability of SLAM in domains where visual processing alone is not sufficient.

Road model prediction based unstructured road detection

Vision-based road detection is an important research topic in different areas of computer vision such as the autonomous navigation of mobile robots. In outdoor unstructured environments such as villages and deserts, the roads are usually not well-paved and have variant colors or texture distributions. Traditional region- or edge-based approaches, however, are effective only in specific environments, and most of them have weak adaptability to varying road types and appearances. In this paper we describe a novel top-down based hybrid algorithm which properly combines both region and edge cues from the images. The main difference between our proposed algorithm and previous ones is that, before road detection, an off-line scene classifier is efficiently learned by both low- and high-level image cues to predict the unstructured road model. This scene classification can be considered a decision process which guides the selection of the optimal solution from region- or edge-based approaches to detect the road. Moreover, a temporal smoothing mechanism is incorporated, which further makes both model prediction and region classification more stable. Experimental results demonstrate that compared with traditional region- and edge-based algorithms, our algorithm is more robust in detecting the road areas with diverse road types and varying appearances in unstructured conditions.

Virtual Environment-Based Teleoperation of Forestry Machines: Designing Future Interaction Methods

Virtual environment-assisted teleoperation has great potential as a human-robot interaction paradigm for field robotic systems, in particular when combined with elements of automation. Unstructured outdoor environments present a complex problem with many challenging elements. For the specific application of forestry machines, we investigate which steps are required in order to implement such a system, what potential benefits there are, and how individual components can be adapted to efficiently assist forestry machine operators in their daily work in the near future. An experimental prototype of a teleoperation system with virtual environment-based feedback is constructed using a scenario-based design process. The feasibility of the implementation is partly verified through experimental studies.

Use of pattern classification algorithms to interpret acoustic echolocation data from a walking-speed robotic sensor platform

An unresolved issue for autonomous walking-speed robots in unstructured outdoor environments is maintaining situational awareness. One strategy is combining information from different sensors so the robot can function in a variety of conditions and environments. The very low-cost Microsoft Kinect accessory incorporates active infrared and RGB video sensors to provide real-time depth information, as well as a 4-channel microphone array. We validate the Kinect sensors and investigate the combination of infrared (passive and active) and non-linear acoustic echolocation sensors on our mobile robotic sensor platform. By using an acoustic parametric array to generate the audible echolocation signal, a tightly-controlled beam of low-frequency sound can interrogate targets at long distances, while infrared imaging works well in the nearfield and in difficult weather conditions. Sophisticated signal processing techniques are required to combine and interpret the collected data; we present an example using pattern ...

Experimental Validation for CRFNFP Algorithm

In 2010,we proposed CRFNFP[1] algorithm to enhance long-range terrain perception for outdoor robots through the integration of both appearance features and spatial contexts. And our preliminary simulation results indicated the superiority of CRFNFP over other existing approaches in terms of accuracy, robustness and adaptability to dynamic unstructured outdoor environments. In this paper, we further study on the comparison experiments for navigation behaviors of robotic systems with different scene perception algorithms in real outdoor scenes. We implemented 3 robotic systems and repeated the running jobs under various conditions. We also defined 3 creterion to facilitate comparison for all systems: Obstacle Response Distance (ORD), Time to Finish Job (TFJ), Distance of the Whole Run (DWR). The comparative experiments indicate that, the CRFNFP-based navigating system outperforms traditional local-map-based navigating systems in terms of all criterion. And the results also show that the CRFNFP algorithm does enhance the long-range perception for mobile robots and helps planning more efficient paths for the navigation.

Extreme navigation: introduction to the special issue

Sophisticated mobile devices with high-quality location sensing technologies are ubiquitous in society. These devices can determine their location and orientation to an accuracy of a few metres. However, the interfaces to such devices are still firmly rooted in the techniques developed for car-based navigation. The user is presented with a map and a dot, or asked to enter a destination, and the device calculates the most efficient route to that location, perhaps with specific written or spoken directions. Whilst these turn-by-turn navigation systems work well for driving, where the user is likely to have a clear destination in mind, they are less use for the myriad of other navigation and wayfinding scenarios that humans undertake on foot. The papers presented in this special issue seek to move navigation beyond simple turn-by-turn directions by embracing the vagaries, vagueness and playfulness that humans often exhibit when navigating on foot. One common theme through the papers presented is the role of undirected or unguided navigation. Magnusson et al. consider how low-fidelity tactile feedback can be employed to indicate the direction of features and facilities in the environment as the user moves his or her device around. Here, the user may have a destination but does not care about the length or time taken to reach it. Specifically, they investigate how different target sizes and signal quality affect the performance of users trying to locate those targets. Their goal is to develop guidelines to allow ‘plug and play’ inclusion of low-fidelity feedback for navigation. Robinson et al. extend upon this work by developing ways to allow these simple directional cues to be adapted to help route users around obstacles when they come into contact with them. By increasing the angular width of feedback, Robinson et al. are able to support users in navigating to a location but allow the user to take a number of different routes, depending on their choices and preferences at the time, to reach that goal. In addition, Robinson et al. found that considering the density and volume of geo-located social media allowed users to be presented with more diverse options for navigation. Whilst both Magnusson et al. and Robinson et al. consider more practical navigation towards an end point, irrespective of the actual route taken to get there, VazquezAlvarez et al. consider more artistic aspects of navigation, augmenting the physical world with geo-located sound. Their auditory sound garden seeks to embrace the wandering behaviour of pedestrians by attracting and pulling them towards interesting sights and features in a public park. They found that the application of spatialised audio provided for greater immersion in the environment and understanding about park features that would otherwise be ignored by pedestrians. In contrast to these unstructured navigation scenarios, Sarjakoski et al. have developed techniques to allow for successful navigation in unstructured outdoor environments. They note that whilst landmark-based navigation (such as discussed by the three previous papers) is important, what constitutes a landmark is not clear. This is especially the case for outdoor hiking navigation, where landmarks are few and far between. Their work provides an important step on classifying the importance of Supported by EU FP7 Project No. 224675 ‘‘Haptimap’’.