Navigation Of Micro Aerial Vehicles Research Articles

Developing an expansion‐based obstacle detection using panoptic segmentation

AbstractSafe Micro Aerial Vehicle (MAV) navigation requires detecting and avoiding obstacles. For safe MAV navigation, expansion‐based algorithms are effective for detecting obstacles. However, accurate and real‐time obstacle detection is a fundamental challenge. Some traditional methods focus on extracting geometric features from images and applying geometric constraints to identify potential obstacles. Others may leverage machine learning algorithms for object detection and classification, using features such as texture, shape, and context to distinguish obstacles from background clutter. The choice of approach depends on factors such as the specific requirements of the application, the complexity of the scene, and the available computational resources. Since obstacles, in reality, take the form of objects (e.g., persons, walls, pillars, trees, automobiles, and other structures), it is preferable to represent them according to human comprehension and as objects. Therefore, the objective of this study is to reflect on the previous research and address the issues mentioned above by extracting objects from the fisheye image using a panoptic deep‐learning network. The extracted object regions are, then, used to identify obstacles with a novel area‐based expansion rate we developed in a previous study. We compared the accuracy of obstacle detection in our proposed method to the existing method when moving forward and to the right; thus, we improved it between 10% and 18%, respectively. In addition, compared with the existing method, and due to replacing a single object with multiple regions, obstacle‐detection runtime for forward and right direction is 15.71 and 25.5 times faster, respectively, and the required match points have decreased by 49% and 55%.

Automated vision-based structural health inspection and assessment for post-construction civil infrastructure

In the search to enhance the effectiveness and efficiency of structural health monitoring for preserving safety and integrity in civil infrastructure, the search for innovative technologies is of critical importance. This paper introduces ExpoDet, a comprehensive framework designed for autonomous health inspection and infrastructure assessment. ExpoDet features a multi-detection detector, autonomous navigation for micro aerial vehicles facilitated through secondary reward reinforcement learning, and a damage aggregation scheme for autonomous health assessment following detections. Moreover, it presents an attention module called EEAM+ that introduces dynamic feature orientation and significantly enhances the capabilities of ExpoDet. ExpoDet is extensively tested and evaluated in both offsite and field test experiments. Comparisons with several state-of-the-art object detectors coupled with attention modules show an average improvement of approximately 3% across various evaluation metrics.

Efficient Object Detection for Micro Aerial Vehicle Navigation Using Enhanced SSD-HOG Descriptors

Autonomous robots, such as micro aerial vehicles (MAVs), require object detection for navigation and inspection tasks. However, the limited computational resources and real-time constraints of MAVs make object detection challenging. To address this, we propose an efficient object detection method for MAVs using enhanced SSD-HOG descriptors. Our method combines HOG and SSD techniques to create enhanced descriptors that provide better object detection accuracy and efficiency than traditional HOG or SSD descriptors. We evaluate our method on an aerial image dataset and compare it with state-of-the-art methods like SSD and YOLO. Our experimental results demonstrate that our method achieves high accuracy and real-time performance while using limited computational resources. Our proposed method is ideal for MAV navigation applications that require real-time object detection with limited computational resources.

REF: A Rapid Exploration Framework for Deploying Autonomous MAVs in Unknown Environments

Exploration and mapping of unknown environments is a fundamental task in applications for autonomous robots. In this article, we present a complete framework for deploying Micro Aerial Vehicles (MAVs) in autonomous exploration missions in unknown subterranean areas. The main motive of exploration algorithms is to depict the next best frontier for the MAV such that new ground can be covered in a fast, safe yet efficient manner. The proposed framework uses a novel frontier selection method that also contributes to the safe navigation of autonomous MAVs in obstructed areas such as subterranean caves, mines, and urban areas. The framework presented in this work bifurcates the exploration problem in local and global exploration. The proposed exploration framework is also adaptable according to computational resources available onboard the MAV which means the trade-off between the speed of exploration and the quality of the map can be made. Such capability allows the proposed framework to be deployed in subterranean exploration and mapping as well as in fast search and rescue scenarios. The performance of the proposed framework is evaluated in detailed simulation studies with comparisons made against a high-level exploration-planning framework developed for the DARPA Sub-T challenge as it will be presented in this article.

RAST: Risk-Aware Spatio-Temporal Safety Corridors for MAV Navigation in Dynamic Uncertain Environments

Autonomous navigation of Micro Aerial Vehicles (MAVs) in dynamic and unknown environments is a complex and challenging task. Current works rely on assumptions to solve the problem. The MAV's pose is precisely known, the dynamic obstacles can be explicitly segmented from static ones, their number is known and fixed, or they can be modeled with given shapes. In this letter, we present a method for MAV navigation in dynamic uncertain environments without making any of these assumptions. The method employs a particle-based dynamic map to represent the local environment and predicts it to the near future. Collision risk is defined based on the predicted maps and a series of risk-aware spatio-temporal (RAST) safety corridors are constructed, which are finally used to optimize a dynamically-feasible collision-free trajectory for the MAV. We compared our method with several state-of-the-art works in 12000 simulation tests in Gazebo with the physical engine enabled. The results show that our method has the highest success rate at different uncertainty levels. Finally, we validated the proposed method in real experiments.

Robust and Efficient Velocity Estimation for MAVs With an RGB-D Camera

Micro Aerial Vehicles (MAVs) have become increasingly popular due to their wide applications in many areas. For autonomous navigation and safe control of MAVs, it is essential to have accurate and reliable velocity and position estimation. However, due to limited computational power and payload, it is still challenging for autonomous operation of MAVs in complex environments. In this paper, we propose a robust and efficient velocity estimation framework for MAVs with a single downward-facing RGB-D camera, which is able to provide metric velocity estimation in three dimensions as well as yaw rate in real-time without the fusion of additional sensors. Unlike traditional algorithms, our method uses kernel cross-correlators (KCCs) to efficiently determine optical flow for motion estimation, which does not rely on costly feature extraction and matching process. Moreover, we utilize depth images to estimate the vertical velocity of MAVs without the assumption of a flat ground. Autonomous flight tests on a quadrotor in complex environments demonstrate the robustness and efficiency of our method.

Vision Object-Oriented Augmented Sampling-Based Autonomous Navigation for Micro Aerial Vehicles

Autonomous navigation of micro aerial vehicles in unknown environments not only requires exploring their time-varying surroundings, but also ensuring the complete safety of flights at all times. The current research addresses estimation of the potential exploration value neglect of safety issues, especially in situations with a cluttered environment and no prior knowledge. To address this issue, we propose a vision object-oriented autonomous navigation method for environment exploration, which develops a B-spline function-based local trajectory re-planning algorithm by extracting spatial-structure information and selecting temporary target points. The proposed method is evaluated in a variety of cluttered environments, such as forests, building areas, and mines. The experimental results show that the proposed autonomous navigation system can effectively complete the global trajectory, during which an appropriate safe distance could always be maintained from multiple obstacles in the environment.

Observability-Constrained VINS for MAVs Using Interacting Multiple Model Algorithm

This article presents the design of an interacting multiple model (IMM) filter for visual-inertial navigation (VIN) of MAVs. VIN of MAVs in practice typically uses a single system model for its state estimator design. However, microaerial vehicles (MAVs) can operate in different scenarios such as aggressive flights, hovering flights, and under high external disturbance requiring changing constraints imposed on the estimator model. This article proposes the use of a conventional VIN and a drag force VIN in an error-state IMM filtering framework to address the need for multiple models in the estimator. The work uses an epipolar geometry constraint for the design of the measurement model for both filters to realize computationally efficient state updates. Observability of the proposed modifications to VIN filters (drag force model and epipolar measurement model) are analyzed, and observability-based consistency rules are derived for the two filters of the IMM. Monte Carlo numerical simulations validate the performance of the observability constrained IMM, which improved the accuracy and consistency of the visual-inertial navigation system (VINS) for changing flight conditions and external wind disturbance scenarios. Experimental validation is performed using the EuRoC dataset to evaluate the performance of the proposed IMM filter design. The results show that the IMM outperforms stand-alone filters used in the IMM filtering bank by switching between the filters based on the residual likelihood of the models.

A Unified NMPC Scheme for MAVs Navigation With 3D Collision Avoidance Under Position Uncertainty

This article proposes a novel Nonlinear Model Predictive Control (NMPC) framework for Micro Aerial Vehicle (MAV) autonomous navigation in constrained environments. The introduced framework allows us to consider the nonlinear dynamics of MAVs and guarantees real-time performance. Our first contribution is to design a computationally efficient subspace clustering method to reveal from geometrical constraints to underlying constraint planes within a 3D point cloud, obtained from a 3D lidar scanner. The second contribution of our work is to incorporate the extracted information into the nonlinear constraints of NMPC for avoiding collisions. Our third contribution focuses on making the controller robust by considering the uncertainty of localization and NMPC using the Shannon entropy. This step enables us to track either the position or velocity references, or none of them if necessary. As a result, the collision avoidance constraints are defined in the local coordinates of MAVs and it remains active and guarantees collision avoidance, despite localization uncertainties, e.g., position estimation drifts. Additionally, as the platform continues the mission, this will result in less uncertain position estimations, due to the feature extraction and loop closure. The efficacy of the suggested framework has been evaluated using various simulations in the Gazebo environment.

Onboard plane-wise 3D mapping using super-pixels and stereo vision for autonomous flight of a hexacopter

Micro Aerial Vehicles (MAVs) have become very popular with a wide range of applications in cinema, surveillance, search and rescue, among many others. For these applications, an experienced pilot could operate the vehicle while keeping it in the line-of-sight. However, if the MAV flies beyond, the MAV may be highly compromised leading to potential crashes due to undetected obstacles. Hence, an assistance system to aid with obstacle detection and avoidance is highly desirable. This problem becomes more evident for autonomous navigation of MAVs. Nevertheless, autonomous navigation involves intensive computational power that rests on additional hardware to that of the flight controller. In addition, MAVs have an inherent limitation in their payload, restricting the size and weight of the onboard hardware and consequently the computational processing and storage capabilities. Motivated by the limitations in hardware, we propose a method to build an high-level map with enough details to make it useful for autonomous navigation, detecting surfaces that can be modeled as planes, thus enabling real-time operation while running on low budget hardware. Our proposed method carries out the following stages: (a) super-pixel segmentation, (b) planes extraction, (c) real-time mapping and (d) RRT navigation. We tested the proposed method in a custom-made hexacopter capable of carrying a stereoscopic camera and a light computer. We show that our MAV navigates in an unknown environment and successfully senses and avoids potential obstacles in its path.

Subterranean MAV Navigation based on Nonlinear MPC with Collision Avoidance Constraints

Micro Aerial Vehicles (MAVs) navigation in subterranean environments is gaining attention in the field of aerial robotics, however there are still multiple challenges for collision free navigation in such harsh environments. This article proposes a novel baseline solution for collision free navigation with Nonlinear Model Predictive Control (NMPC). In the proposed method, the MAV is considered as a floating object, where the velocities on the x, y axes and the position on altitude are the references for the NMPC to navigate along the tunnel, while the NMPC avoids the collision by considering kinematics of the obstacles based on measurements from a 2D lidar. Moreover, a novel approach for correcting the heading of the MAV towards the center of the mine tunnel is proposed, while the efficacy of the suggested framework has been evaluated in multiple field trials in an underground mine in Sweden.

Indoor versus outdoor scene recognition for navigation of a micro aerial vehicle using spatial color gist wavelet descriptors

In the context of improved navigation for micro aerial vehicles, a new scene recognition visual descriptor, called spatial color gist wavelet descriptor (SCGWD), is proposed. SCGWD was developed by combining proposed Ohta color-GIST wavelet descriptors with census transform histogram (CENTRIST) spatial pyramid representation descriptors for categorizing indoor versus outdoor scenes. A binary and multiclass support vector machine (SVM) classifier with linear and non-linear kernels was used to classify indoor versus outdoor scenes and indoor scenes, respectively. In this paper, we have also discussed the feature extraction methodology of several, state-of-the-art visual descriptors, and four proposed visual descriptors (Ohta color-GIST descriptors, Ohta color-GIST wavelet descriptors, enhanced Ohta color histogram descriptors, and SCGWDs), in terms of experimental perspectives. The proposed enhanced Ohta color histogram descriptors, Ohta color-GIST descriptors, Ohta color-GIST wavelet descriptors, SCGWD, and state-of-the-art visual descriptors were evaluated, using the Indian Institute of Technology Madras Scene Classification Image Database two, an Indoor-Outdoor Dataset, and the Massachusetts Institute of Technology indoor scene classification dataset [(MIT)-67]. Experimental results showed that the indoor versus outdoor scene recognition algorithm, employing SVM with SCGWDs, produced the highest classification rates (CRs)—95.48% and 99.82% using radial basis function kernel (RBF) kernel and 95.29% and 99.45% using linear kernel for the IITM SCID2 and Indoor-Outdoor datasets, respectively. The lowest CRs—2.08% and 4.92%, respectively—were obtained when RBF and linear kernels were used with the MIT-67 dataset. In addition, higher CRs, precision, recall, and area under the receiver operating characteristic curve values were obtained for the proposed SCGWDs, in comparison with state-of-the-art visual descriptors.

Autonomous navigation of micro aerial vehicles using high-rate and low-cost sensors

The combination of visual and inertial sensors for state estimation has recently found wide echo in the robotics community, especially in the aerial robotics field, due to the lightweight and complementary characteristics of the sensors data. However, most state estimation systems based on visual-inertial sensing suffer from severe processor requirements, which in many cases make them impractical. In this paper, we propose a simple, low-cost and high rate method for state estimation enabling autonomous flight of micro aerial vehicles, which presents a low computational burden. The proposed state estimator fuses observations from an inertial measurement unit, an optical flow smart camera and a time-of-flight range sensor. The smart camera provides optical flow measurements up to a rate of 200 Hz, avoiding the computational bottleneck to the main processor produced by all image processing requirements. To the best of our knowledge, this is the first example of extending the use of these smart cameras from hovering-like motions to odometry estimation, producing estimates that are usable during flight times of several minutes. In order to validate and defend the simplest algorithmic solution, we investigate the performances of two Kalman filters, in the extended and error-state flavors, alongside with a large number of algorithm modifications defended in earlier literature on visual-inertial odometry, showing that their impact on filter performance is minimal. To close the control loop, a non-linear controller operating in the special Euclidean group SE(3) is able to drive, based on the estimated vehicle’s state, a quadrotor platform in 3D space guaranteeing the asymptotic stability of 3D position and heading. All the estimation and control tasks are solved on board and in real time on a limited computational unit. The proposed approach is validated through simulations and experimental results, which include comparisons with ground-truth data provided by a motion capture system. For the benefit of the community, we make the source code public.

Multi-camera visual SLAM for autonomous navigation of micro aerial vehicles

In this paper, we present a visual simultaneous localization and mapping (SLAM) system which integrates measurements from multiple cameras to achieve robust pose tracking for autonomous navigation of micro aerial vehicles (MAVs) in unknown complex environments. We analyze the iterative optimizations for pose tracking and map refinement of visual SLAM in multi-camera cases. The analysis ensures the soundness and accuracy of each optimization update. A well-known monocular visual SLAM system is extended to utilize two cameras with non-overlapping fields of view (FOVs) in the final implementation. The resulting visual SLAM system enables autonomous navigation of an MAV in complex scenarios. The theory behind this system can easily be extended to multi-camera configurations, when the onboard computational capability allows this. For operations in large-scale environments, we modify the resulting visual SLAM system to be a constant-time robust visual odometry. To form a full visual SLAM system, we further implement an efficient back-end for loop closing. The back-end maintains a keyframe-based global map, which is also used for loop-closure detection. An adaptive-window pose-graph optimization method is proposed to refine keyframe poses of the global map and thus correct pose drift that is inherent in the visual odometry. We demonstrate the efficiency of the proposed visual SLAM system for applications onboard of MAVs in experiments with both autonomous and manual flights. The pose tracking results are compared with ground truth data provided by an external tracking system.

Vision-Based Heading and Lateral Deviation Estimation for Indoor Navigation of a Quadrotor

ABSTRACT In this paper, an effective vanishing point detection method based on grid-based approach for navigation of micro aerial vehicles (MAVs) in indoor corridor environment is described. The automation and navigation of MAVs in GPS-denied indoor environments is still a challenging task. To navigate in indoor corridors, on-board camera sensor is used as a sole sensor for estimating both the vehicle position and heading. Finally, the lateral deviation and heading of an MAV from the centre towards the right or left of the corridor are computed using vanishing point coordinates between two successive frames. By exploiting geometrical constraints inside buildings, long-term stable heading information is extracted. It is shown that the proposed method effectively eliminates heading drift in indoor corridor environment and is real-time capable. The measured parameters can be used to control an MAV when it deviates from the desired path of navigation.

Micro/Miniature Aerial Vehicle Guidance for Hurricane Research

We present an investigation of the Lagrangian dynamics of a hurricane from the perspective of using micro aerial vehicles (MAVs) or small unmanned aircraft systems as mobile sensors for hurricane research and monitoring. The low cost of MAVs allows the use of more sensors for volumetric in situ measurements, particularly in high-risk locations where larger more expensive systems cannot be used. However, the limited flight speed of MAVs enforces severe restrictions on potential flight plans. In the face of 70-m/s horizontal winds, a MAV with only 10-m/s flight capability has very little directional control. Fortunately, vertical wind speeds in the hurricane are much lower and have limited spatial extent. Using Lagrangian coherent structure techniques and simplified vehicle simulations, we develop an understanding of the transport dynamics of a simulated hurricane and apply this understanding to a high-level control scheme to enable MAV navigation and guidance near the hurricane. By smartly adjusting their altitude, simulated MAVs are able to navigate into the hurricane eye at a very high success rate. Our findings suggest that the smart use of the existing background flow could allow the use of low-cost sensor platforms in extreme environments.

Exploration and Mapping Technique Suited for Visual-features Based Localization of MAVs

An approach for long term localization, stabilization, and navigation of micro-aerial vehicles (MAVs) in unknown environment is presented in this paper. The proposed method relies strictly on onboard sensors of employed MAVs and does not require any external positioning system. The core of the method consists in extraction of information from pictures consequently captured using a camera carried by the particular MAV. Visual features are obtained from images of the surface under the MAV, and stored into a map that is represented by these features. The position of the MAV is then obtained through matching with previously stored features. An important part of the proposed system is a novel approach for exploration and mapping of the workspace of robots. This method enables efficient exploring of the unknown environment, while keeping the iteratively built map of features consistent. The proposed algorithm is suitable for mapping of surfaces, both outdoor and indoor, with various density of the image features. The sufficient precision and long term persistence of the method allows its utilization for stabilization of large MAV groups that work in formations with small relative distances between particular vehicles. Numerous experiments with quadrotor helicopters and various numerical simulations have been realized for verification of the entire system and its components.

Autonomous Navigation for Micro Aerial Vehicles in Complex GNSS-denied Environments

Micro aerial vehicles, such as multirotors, are particular well suited for the autonomous monitoring, inspection, and surveillance of buildings, e.g., for maintenance in industrial plants. Key prerequisites for the fully autonomous operation of micro aerial vehicles in restricted environments are 3D mapping, real-time pose tracking, obstacle detection, and planning of collision-free trajectories. In this article, we propose a complete navigation system with a multimodal sensor setup for omnidirectional environment perception. Measurements of a 3D laser scanner are aggregated in egocentric local multiresolution grid maps. Local maps are registered and merged to allocentric maps in which the MAV localizes. For autonomous navigation, we generate trajectories in a multi-layered approach: from mission planning over global and local trajectory planning to reactive obstacle avoidance. We evaluate our approach in a GNSS-denied indoor environment where multiple collision hazards require reliable omnidirectional perception and quick navigation reactions.

EVALUATION OF STEREO ALGORITHMS FOR OBSTACLE DETECTION WITH FISHEYE LENSES

Abstract. For autonomous navigation of micro aerial vehicles (MAVs), a robust detection of obstacles with onboard sensors is necessary in order to avoid collisions. Cameras have the potential to perceive the surroundings of MAVs for the reconstruction of their 3D structure. We equipped our MAV with two fisheye stereo camera pairs to achieve an omnidirectional field-of-view. Most stereo algorithms are designed for the standard pinhole camera model, though. Hence, the distortion effects of the fisheye lenses must be properly modeled and model parameters must be identified by suitable calibration procedures. In this work, we evaluate the use of real-time stereo algorithms for depth reconstruction from fisheye cameras together with different methods for calibration. In our experiments, we focus on obstacles occurring in urban environments that are hard to detect due to their low diameter or homogeneous texture.

Automatic detection of complex shaped buildings in aerial images to support the navigation of micro aerial vehicles in urban environment

In areas with insufficient GPS reception, like in urban areas close to buildings, alternative techniques have to be used to assist the inertial navigation system. The long-term objective of this work is to use buildings, detected in camera images, as distinctive landmarks for navigating micro aerial vehicles within the aforementioned areas. This paper presents a new method to detect buildings in aerial images. To use this algorithm onboard the vehicle during the mission, it has to be fast and executed automatically without readjusting any parameters by the operator. To cover a wide range of possible application areas, no building constraints are required. Therefore a wide variation of buildings with complex shapes can be detected.