Vision-based Landing Research Articles

Vision-Based Marker-Less Landing of an Unmanned Aerial System on Moving Ground Vehicle

Current autonomous unmanned aerial systems (UASs) commonly use vision-based landing solutions that depend upon fiducial markers to localize a static or mobile landing target relative to the UAS. This paper develops and demonstrates an alternative method to fiducial markers with a combination of neural-network-based object detection and camera intrinsic properties to localize an unmanned ground vehicle (UGV) and enable autonomous landing. Implementing this visual approach is challenging given the limited compute power on board the UAS, but it is relevant for autonomous landings on targets for which affixing a fiducial marker a priori is not possible or not practical. The position estimate of the UGV is used to formulate a landing trajectory that is then input to the flight controller. Algorithms are tailored toward low size, weight, and power constraints, as all compute and sensing components weigh less than 100 g. Landings were successfully demonstrated in both simulation and experimentally on a UGV traveling in both a straight line and while turning. Simulation landings were successful at UGV speeds of up to 3.0 m/s, and experimental landings at speeds up to 1.0 m/s.

Vision-Based UAV Landing with Guaranteed Reliability in Adverse Environment

Safe and accurate landing is crucial for Unmanned Aerial Vehicles (UAVs). However, it is a challenging task, especially when the altitude of the landing target is different from the ground and when the UAV is working in adverse environments, such as coasts where winds are usually strong and changing rapidly. UAVs controlled by traditional landing algorithms are unable to deal with sudden large disturbances, such as gusts, during the landing process. In this paper, a reliable vision-based landing strategy is proposed for UAV autonomous landing on a multi-level platform mounted on an Unmanned Ground Vehicle (UGV). With the proposed landing strategy, visual detection can be retrieved even with strong gusts and the UAV is able to achieve robust landing accuracy in a challenging platform with complex ground effects. The effectiveness of the landing algorithm is verified through real-world flight tests. Experimental results in farm fields demonstrate the proposed method’s accuracy and robustness to external disturbances (e.g., wind gusts).

Long Endurance Site-Specific Management of Biochar Applications Using Unmanned Aircraft Vehicle and Unmanned Ground Vehicle

Agricultural activities emit an increasing amount of carbon into the atmosphere, causing climate change. Carbon sequestration is one of the carbon-neutral technologies for reducing carbon emissions by spraying biochar in needed places, such as fertilizer places. It is critical to monitor carbon emissions and the use of biochar in order to properly manage carbon emissions. Prior methods, such as traditional soil or manure sample collection and semiautomatic analysis, are expensive and time-consuming, and they cannot make decisions in real time. We propose long-term, site-specific biochar application management using unmanned aircraft vehicles and unmanned ground vehicles equipped with proximity radar sensing. Our system takes the strategy that a UAV will land on a moving UGV to acquire long-range information with a reliable backup. Meanwhile, the UGV has many payload options to supply the UAV's sensing and actuation missions. We evaluated our system by using a simulated mission approach that contains the analysis of the controller for the landing sequence, and real-world vision-based landing marker tracking. Results show that our system achieves robustness landing control sequence and the error altitude calculation based on computer vision is less than ± 0.02 m on average.

Low-Complexity Control for Vision-Based Landing of Quadrotor UAV on Unknown Moving Platform

This article addresses the vision-based landing problem of a low-cost quadrotor on an unknown moving platform. A robust landing controller is developed, which consists of the design of the low-complexity outer-loop controller and the geometric inner-loop attitude controller. First, an error transformation based on prescribed performance is designed to guarantee the landing behaviors and deal with the intermediate control signal of backstepping approaches, resulting in a low-complexity position-based visual servoing (PBVS) design. In addition, the proposed PBVS controller exhibits strong robustness against an uncertain relative dynamic system due to no incorporation of any prior knowledge of the moving platform. Next, a modified geometric attitude controller is presented by characterizing the geometric properties of rotation matrices intrinsically. Finally, the stability analysis is presented using Lyapunov stability theory, and the effectiveness of the proposed control strategy is demonstrated through numerical simulations and experiments.

Rotorcraft Dynamic Platform Landings Using Robotic Landing Gear

Abstract Articulating landing gear that use closed-loop feedback control are proven to expand the landing capabilities of rotorcraft on sloped and rough terrain. These systems are commonly referred to as robotic landing gear (RLG). Modern RLG systems have limitations for landing on dynamic platforms because their controllers do not incorporate fuselage roll and roll rate feedback. This work presents a proven crashworthy cable-driven RLG system for the commercial S-100 Camcopter that expands static landing zone limits by a factor of three and enables dynamic platform landings in rough sea state (SS) conditions. A new roll and foot-force feedback fused control algorithm is developed to enable ship deck landings of an RLG equipped S-100 without the need for deck lock or advanced vision-based landing systems. Multibody dynamic simulations of the aircraft, landing gear, and new control system show the benefits of this combined roll and force feedback approach. Results include experimental dynamic landings on platforms rolling under sinusoidal motion and simulated SS conditions. The experiments demonstrate, in a limited fashion, the usability of the RLG through ground experimentation, and the results are compared to simulations. Additional simulations of landings of the S-100 with rigid and active landing gear with more challenging landing conditions than experimentally tested are presented. Such results aid in understanding how RLG with this new roll and contact force fused controller prevent dynamic rollover.

운송드론을 위한 영상기반 착륙점 추적알고리즘 연구

본 논문은 운송드론의 영상기반 착륙을 위한 착륙점 영상추적 알고리즘 적용연구에 관한 것이다. 현재까지 대부분의 연구개발에서 운송드론은 GPS같은 위성항법 기반으로 착륙을 하고 있으나, 착륙시의 안전을 고려하여 운용자가 영상으로 안전여부를 확인하고 착륙지점을 지정하는 운용 방안이 필요하다고 본다. 본 연구에서는 운송드론이 착륙지점에 다가가며 운용자가 드론이 보내온 영상에서 착륙점을 지정하고, 이를 추적하여 드론이 착륙하는 운용개념을 가정하고, 이에 적합한 세 종류의 영상추적기 (MOSSE, KCF, DSST)에 대한 성능비교연구 결과를 기술한다. 특히 착륙지점 표식의 크기가 드론이 다가감에 따라 크게 변하는 특징과 추적기의 실시간성을 고려한 영상추적기의 패러미터 설정에 대한 방안을 제안한다. MOSSE와 KCF필터의 경우 추적창 크기가 고정되어 있어 드론이 착륙점에 다가가며 영상추적 필터링 시간의 변화가 거의 없으나, DSST 알고리즘의 경우 최대추적창 크기 패러미터를 480픽셀에서 60픽셀로 변화시키면 필터링 속도가 최대 약 8배 빨라진다. 그리고 최대추적창 크기 패러미터가 120픽셀일 때 추적성능과 필터링시간이 우수함을 보여준다. 상용드론으로 획득한 비행영상을 활용하여 각 영상추적기 성능평가 결과를 보이고, 향후 운송드론에 본 연구에서 수행한 영상추적기를 어떻게 적용할지에 대한 방향을 제시한다.

A Vision-based navigation system for landing procedure

An autonomous vision-based landing system was designed and its performance is analysed and measured by an UAS. The system equipment is based on a single camera to determine its position and attitude with respect to a well-defined landing pattern. The developed procedure is based on photogrammetric Space Resection Solution, which provides the position and camera attitude reckoning starting from at least three, not aligned, reference control points whose image coordinates may be measured in the image camera frame. Five circular coloured targets were placed on a specific landing pattern, their 2D image frame coordinates was extracted through a particular algorithm. The aim of this work is to compute UAS precise position and attitude from single image, in order to have a good approach to landing field. This procedure can be used in addition or for replacement of GPS tracking and can be applied when the landing field is movable or located on a moving platform, the UAS will follow the landing pattern until the landing phase will be closed.

Finite-Time LPV Analysis of a Vision Based Landing System with Anti-Windup Augmentation

Abstract In contrast to standard ILS-based landing control systems, vision-based strategies rely on specific measures that introduce non-linearities into the control laws. Moreover, during the landing phase high gains are often used to improve performances at the expense of stability which is then degraded. This is not much an issue since both final approach and flare segments are short-time maneuvers. However, high gains induce saturations which in turn lead to performances degradation. For these reasons, vision-based landing systems design and tuning require a specific attention which generally involves a time-consuming trial-and-error process. The latter could certainly be fasten and improved with the help of adapted analysis tools providing a reliable estimate of the finite-time performance level of the aforementioned saturated nonlinear closed-loop system. As observed in Biannic and Burlion (2017), it can be rewritten as a saturated Linear Parameter Varying (LPV) system for which many tools exist or can be extended. In this contribution, original discrete-time extensions are then proposed to characterize finite-time performance indexes for a vision-based landing system including an anti-windup device.

Performance analysis of saturated parameter-varying systems with application to vision-based landing assessment

Unlike standard landing control laws, vision-based systems exploit specific measurements which, as shown in this paper, introduce a rapidly growing parameter in the closed-loop system to be evaluated. Moreover, during the landing phase, high gains are often used to ensure performance despite perturbations. As a result, saturations are likely to appear. Based on parameter-dependent Lyapunov functions, an efficient algorithm is proposed in this paper to characterize stability and performance of parameter-varying systems with control input saturations. The algorithm is then successfully applied to the evaluation process of a simple vision-based landing system.

Marker-guided auto-landing on a moving platform

PurposeThe purpose of this paper is to develop a method for a vision-based automatic landing of a multi-rotor unmanned aerial vehicle (UAV) on a moving platform. The landing system must be highly accurate and meet the size, weigh, and power restrictions of a small UAV.Design/methodology/approachThe vision-based landing system consists of a pattern of red markers placed on a moving target, an image processing algorithm for pattern detection, and a servo-control for tracking. The suggested approach uses a color-based object detection and image-based visual servoing.FindingsThe developed prototype system has demonstrated the capability of landing within 25 cm of the desired point of touchdown. This auto-landing system is small (100×100 mm), light-weight (100 g), and consumes little power (under 2 W).Originality/valueThe novelty and the main contribution of the suggested approach are a creative combination of work in two fields: image processing and controls as applied to the UAV landing. The developed image processing algorithm has low complexity as compared to other known methods, which allows its implementation on general-purpose low-cost hardware. The theoretical design has been verified systematically via simulations and then outdoors field tests.

Feature article: Overview of landmarks for autonomous, vision-based landing of unmanned helicopters

The characteristics of a landing landmark are a key concern when an autonomous, vision-based unmanned helicopter is landing. The shape and recognition of the landmark determines the posture parameters of the unmanned helicopter and the accuracy of its autonomous landing. This article analyzes and summarizes the shapes and the detection methods of various landmarks used for autonomous landing of helicopters, as well as their relationship to the posture parameters. The characteristics of different landing landmarks are obtained, and then the development of autonomous landing landmarks is analyzed. Finally, this article offers some directions for the future research of autonomous, vision-based landing of unmanned helicopters.

Landing Control on a Mobile Platform for Multi-copters using an Omnidirectional Image Sensor

This paper proposes a vision-based target following and landing system for a quadrotor vehicle on a moving platform. The system employs a vision-based landing site detection and locating algorithm using an omnidirectional lens. The latest smartphone was attached to the UAV and served as an on-board image acquisition and process unit. Measurements from the omnidirectional camera are combined with a proper dynamic model in order to estimate position and velocity of the moving platform. An adaptive control scheme was implemented on the flight computer to deal with unknown disturbances in outdoor environments. The system was validated on a quadrotor UAV and the vehicle successfully landed on the moving platform during outdoor flight tests.

Autonomous UAV controlled collision landing via eye-in-hand visual servoing

Catastrophic damage to fixed-wing unmanned aerial vehicles (UAV) is most likely to occur during the landing stage. Vision-based autonomous landing methods for small UAV have been proposed in literature to address the issue. This paper improves on previous work on vision-based landing via a controlled collision with a large inflatable airbag. In the proposed method the UAV steers towards a large, multi-coloured, dome-shaped airbag using visual feedback obtained from a UAV-mounted camera. A multi-colour airbag is proposed to facilitate a robust and efficient colour-based detection method for real-time detection of the dome in images. A proposed visual servoing controller translates the dome location into desired pitch and yaw rate commands to achieve an exponential decay of the image error. A model predictive UAV controller that achieves desired roll and yaw rates is designed and tested in simulations. The lateral and longitudinal UAV dynamics were decoupled and a linear discrete state space model of the dynamics was obtained via system identification. The proposed dome landing method has been implemented on the FlightGear flight simulator and has presented successful landings in simulations over multiple trials.

Vision-based landing site evaluation and informed optimal trajectory generation toward autonomous rooftop landing

Autonomous landing is an essential function for micro air vehicles (MAVs) for many scenarios. We pursue an active perception strategy that enables MAVs with limited onboard sensing and processing capabilities to concurrently assess feasible rooftop landing sites with a vision-based perception system while generating trajectories that balance continued landing site assessment and the requirement to provide visual monitoring of an interest point. The contributions of the work are twofold: (1) a perception system that employs a dense motion stereo approach that determines the 3D model of the captured scene without the need of geo-referenced images, scene geometry constraints, or external navigation aids; and (2) an online trajectory generation approach that balances the need to concurrently explore available rooftop vantages of an interest point while ensuring confidence in the landing site suitability by considering the impact of landing site uncertainty as assessed by the perception system. Simulation and experimental evaluation of the performance of the perception and trajectory generation methodologies are analyzed independently and jointly in order to establish the efficacy and robustness of the proposed approach.

Pulse-coupled neural network performance for real-time identification of vegetation during forced landing

Safety concerns in the operation of autonomous aerial systems require safe-landing protocols be followed during situations where the mission should be aborted due to mechanical or other failure. This article presents a pulse-coupled neural network (PCNN) to assist in the vegetation classification in a vision-based landing site detection system for an unmanned aircraft. We propose a heterogeneous computing architecture and an OpenCL implementation of a PCNN feature generator. Its performance is compared across OpenCL kernels designed for CPU, GPU, and FPGA platforms. This comparison examines the compute times required for network convergence under a variety of images to determine the plausibility for real-time feature detection.

A Hierarchical Tracking Strategy for Vision-Based Applications On-Board UAVs

In this paper, we apply a hierarchical tracking strategy of planar objects (or that can be assumed to be planar) that is based on direct methods for vision-based applications on-board UAVs. The use of this tracking strategy allows to achieve the tasks at real-time frame rates and to overcome problems posed by the challenging conditions of the tasks: e.g. constant vibrations, fast 3D changes, or limited capacity on-board. The vast majority of approaches make use of feature-based methods to track objects. Nonetheless, in this paper we show that although some of these feature-based solutions are faster, direct methods can be more robust under fast 3D motions (fast changes in position), some changes in appearance, constant vibrations (without requiring any specific hardware or software for video stabilization), and situations in which part of the object to track is outside of the field of view of the camera. The performance of the proposed tracking strategy on-board UAVs is evaluated with images from real-flight tests using manually-generated ground truth information, accurate position estimation using a Vicon system, and also with simulated data from a simulation environment. Results show that the hierarchical tracking strategy performs better than well-known feature-based algorithms and well-known configurations of direct methods, and that its performance is robust enough for vision-in-the-loop tasks, e.g. for vision-based landing tasks.

Vision-Based Landing of Light Weight Unmanned Helicopters on a Smart Landing Platform

This paper presents a simple and efficient solution to vision guided autonomous landing of a light-weight (<150 Kg) unmanned helicopter on a smart landing platform, called ISLANDS--Intelligent Self-Leveling and Nodal Docking System. The advantage of ISLANDS is that it may allow the helicopter upon docking to recharge its batteries or refuel, thus, indirectly increasing endurance and flight range. In order for the helicopter to dock with ISLANDS, an on-board `vision module' coupled with the helicopter attitude controller is developed. This `vision module' detects the location and orientation of ISLANDS and feeds back information to the helicopter attitude controller, which commands the helicopter to descent onto the landing platform at a desired orientation and speed. The Scale Invariant Feature Transform (SIFT) is used for automatic detection of the landing platform based on images captured by a single camera mounted on the helicopter. The detected SIFT features are used to estimate the 3-D orientation of the platform relative to the helicopter using Homography and RANSAC techniques. The focus of this paper is on the vision-guided landing technique in a predefined orientation and not on controller details, which may be found in Shim et al. (1998).

A vision-based landing system for small unmanned aerial vehicles using an airbag

Statistics show that the landing accounts for the largest portion of all mishaps of unmanned aerial vehicles (UAVs) due to many difficulties including limited situational awareness of the external pilot and the limited maneuverability during the low speed flight before touchdown. In this paper, a vision-based automatic landing system using a dome-shaped airbag is proposed for small UAVs. Its isotropic shape allows airplanes to approach in any direction to avoid crosswind unlike net-assisted landing. The dome’s distinctive color improves the detection owing to its strong visual cue. Color- and shape-based detection vision algorithms are applied for robust detection under varying lighting conditions. Due to the insufficient accuracy of navigation sensors, a direct visual servoing is used for terminal guidance. The proposed algorithm is validated in a series of flight tests.

A Vision-Based Automatic Landing Method for Fixed-Wing UAVs

In this paper, a vision-based landing system for small-size fixed-wing unmanned aerial vehicles (UAVs) is presented. Since a single GPS without a differential correction typically provide position accuracy of at most a few meters, an airplane equipped with a single GPS only is not guaranteed to land at a designated location with a sufficient accuracy. Therefore, a visual servoing algorithm is proposed to improve the accuracy of landing. In this scheme, the airplane is controlled to fly into the visual marker by directly feeding back the pitch and yaw deviation angles sensed by the forward-looking camera during the terminal landing phase. The visual marker is a monotone hemispherical airbag, which serves as the arresting device while providing a strong and yet passive visual cue for the vision system. The airbag is detected by using color- and moment-based target detection methods. The proposed idea was tested in a series of experiments using a blended wing-body airplane and proven to be viable for landing of small fixed-wing UAVs.

Mars and Moon exploration passing through the European Precision Landing GNC Test Facility

In the frame of the ESA Aurora exploration program, the precision landing on Mars and Moon is of strategic importance because the ability to land on sites with high scientific interest is not a straightforward task.From here is the need to provide a dedicated test facility able to test Guidance, Navigation and Control (GNC) techniques and technologies, both vision and LIDAR based, in a realistic environment, in real time and as much as possible in a real scale. This is a fundamental step through which the planetary exploration will pass through in order to reduce the inherent risks associated with any landing mission.The ESA project named Precision Landing GNC Test Facility (PLGTF) has been conceived with this purpose, has been successfully started and, at the time of writing of the paper, has reached the detailed design phase. PLGTF will emulate a real lander dynamics during the powered descent phase in an atmospheric and non-atmospheric planetary environment, minimising the unavoidable scale factor to be applied.PLGTF is going to be the first ever-built facility with such a performance and it will be able to validate the entire GNC chain of a planetary landing system.The paper describes the PLGTF configuration for what concerns its first implementation, mainly devoted to validate the navigation function of a vision-based landing system and the associated sensor, and will also provide some preliminary information on the planned enhancements.