Vision-based Navigation System Research Articles

A Perception-Aware Flatness-Based Model Predictive Controller for Fast Vision-Based Multirotor Flight

Abstract Despite the push toward fast, reliable vision-based multirotor flight, most vision-based navigation systems still rely on controllers that are perception-agnostic. Given that these controllers ignore their effect on the system’s localisation capabilities, they can produce an action that allows vision-based localisation (and consequently navigation) to fail. In this paper, we present a perception-aware flatness-based model predictive controller (MPC) that accounts for its effect on visual localisation. To achieve perception awareness, we first develop a simple geometric model that uses over 12 km of flight data from two different environments (urban and rural) to associate visual landmarks with a probability of being successfully matched. In order to ensure localisation, we integrate this model as a chance constraint in our MPC such that we are probabilistically guaranteed that the number of successfully matched visual landmarks exceeds a minimum threshold. We show how to simplify the chance constraint to a nonlinear, deterministic constraint on the position of the multirotor. With desired speeds of 10 m/s, we demonstrate in simulation (based on real-world perception data) how our proposed perception-aware MPC is able to achieve faster flight while guaranteeing localisation compared to similar perception-agnostic controllers. We illustrate how our perception-aware MPC adapts the path constraint along the path based on the perception model by accounting for camera orientation, path error and location of the visual landmarks. The result is that repeating the same geometric path but with the camera facing in opposite directions can lead to different optimal paths flown.

Improved vehicle navigation using sensor fusion of inertial, odometeric sensors with global positioning system

The primary objective of a navigation system is to continuously monitor the trajectory of a vehicle. Navigation for a land vehicle is implemented using different measurement systems, such as Inertial Navigation System (INS), radio navigation system, vision-based navigation and Global Positioning System (GPS). INS provides continuous information of position, velocity and attitude. However, its performance deteriorates with time since the errors tend to accumulate. GPS is an electromagnetic signal which is more accurate when compared to INS but cannot provide continuous and reliable position all the time. The drawbacks of these individual systems have given rise to the need for higher accuracy, integrity and robustness. This has led to the fusion of measurements from these sensors to obtain an improved performance in measuring the position, velocity of a vehicle. This paper discusses the simulation and implementation of an integrated navigation system using inertial, odometric sensors with GPS using scaled unscented Kalman filter. The method discussed involves obtaining a state transition model and a measurement model of the sensors and processing the states using scaled unscented Kalman filter to obtain better estimates of position and velocity.

Improved vehicle navigation using sensor fusion of inertial, odometeric sensors with global positioning system

The primary objective of a navigation system is to continuously monitor the trajectory of a vehicle. Navigation for a land vehicle is implemented using different measurement systems, such as Inertial Navigation System (INS), radio navigation system, vision-based navigation and Global Positioning System (GPS). INS provides continuous information of position, velocity and attitude. However, its performance deteriorates with time since the errors tend to accumulate. GPS is an electromagnetic signal which is more accurate when compared to INS but cannot provide continuous and reliable position all the time. The drawbacks of these individual systems have given rise to the need for higher accuracy, integrity and robustness. This has led to the fusion of measurements from these sensors to obtain an improved performance in measuring the position, velocity of a vehicle. This paper discusses the simulation and implementation of an integrated navigation system using inertial, odometric sensors with GPS using scaled unscented Kalman filter. The method discussed involves obtaining a state transition model and a measurement model of the sensors and processing the states using scaled unscented Kalman filter to obtain better estimates of position and velocity.

Run-Time Reconfigurable MPSoC-Based On-Board Processor for Vision-Based Space Navigation

This paper describes a reconfigurable architecture for an on-board processor to be used in space exploration critical systems. It relies on, a dynamically reconfigurable multi-accelerator hardware architecture that provides transparent reconfiguration and scalable performance, dependability, and power consumption, at run-time. The architecture is integrated under an RTEMS operating system, which manages reconfiguration and fault mitigation in a fully-compatible way with space requirements. In this work, the proposed processor is used to implement a vision-based navigation system, providing fully autonomous adaptation to different phases of a space mission timeline or events that the spacecraft may encounter during its lifespan. Results show that reconfigurability enables the practical usage of Commercial Off-The-Shelf (COTS) Multiprocessor Systems-on-Chips (MPSoCs) in real scenarios.

An Adaptive Vision Navigation Algorithm in Agricultural IoT System for Smart Agricultural Robots

As the agricultural internet of things (IoT) technology has evolved, smart agricultural robots needs to have both flexibility and adaptability when moving in complex field environments. In this paper, we propose the concept of a vision-based navigation system for the agricultural IoT and a binocular vision navigation algorithm for smart agricultural robots, which can fuse the edge contour and the height information of rows of crop in images to extract the navigation parameters. First, the speeded-up robust feature (SURF) extracting and matching algorithm is used to obtain featuring point pairs from the green crop row images observed by the binocular parallel vision system. Then the confidence density image is constructed by integrating the enhanced elevation image and the corresponding binarized crop row image, where the edge contour and the height information of crop row are fused to extract the navigation parameters (θ, d) based on the model of a smart agricultural robot. Finally, the five navigation network instruction sets are designed based on the navigation angle θ and the lateral distance d, which represent the basic movements for a certain type of smart agricultural robot working in a field. Simulated experimental results in the laboratory show that the algorithm proposed in this study is effective with small turning errors and low standard deviations, and can provide a valuable reference for the further practical application of binocular vision navigation systems in smart agricultural robots in the agricultural IoT system.

An adaptive distributed approach for the real-time vision-based navigation system

Real time navigation has remained as a major challenge for guidance and control of robots in indoor applications. In this paper, an adaptive image processing method is suggested to determine the 3D position and velocity of moving objects by using a distributed camera array. In this method objects are detected and 2D localized by independently and asynchronously processing in each camera. Afterwards, the extracted location data of all cameras are fed into a distributed adaptive extended Kalman filter to calculate the 3D position of multiple targets. Improving some specifications such as accuracy, area coverage, robustness against camera noise and camera failure, sampling time and the total price of the system is the main design objectives. The presented data fusion technique enables real-time localization of moving objects in 3D space by using a low-cost camera array. An experimental setup was used to illustrate the performance of the suggested localization system.

A vision-based navigation system for Unmanned Aerial Vehicles (UAVs)

A vision-based navigation system for Unmanned Aerial Vehicles (UAVs)

Vision-Based Robot Navigation through Combining Unsupervised Learning and Hierarchical Reinforcement Learning.

Extensive studies have shown that many animals’ capability of forming spatial representations for self-localization, path planning, and navigation relies on the functionalities of place and head-direction (HD) cells in the hippocampus. Although there are numerous hippocampal modeling approaches, only a few span the wide functionalities ranging from processing raw sensory signals to planning and action generation. This paper presents a vision-based navigation system that involves generating place and HD cells through learning from visual images, building topological maps based on learned cell representations and performing navigation using hierarchical reinforcement learning. First, place and HD cells are trained from sequences of visual stimuli in an unsupervised learning fashion. A modified Slow Feature Analysis (SFA) algorithm is proposed to learn different cell types in an intentional way by restricting their learning to separate phases of the spatial exploration. Then, to extract the encoded metric information from these unsupervised learning representations, a self-organized learning algorithm is adopted to learn over the emerged cell activities and to generate topological maps that reveal the topology of the environment and information about a robot’s head direction, respectively. This enables the robot to perform self-localization and orientation detection based on the generated maps. Finally, goal-directed navigation is performed using reinforcement learning in continuous state spaces which are represented by the population activities of place cells. In particular, considering that the topological map provides a natural hierarchical representation of the environment, hierarchical reinforcement learning (HRL) is used to exploit this hierarchy to accelerate learning. The HRL works on different spatial scales, where a high-level policy learns to select subgoals and a low-level policy learns over primitive actions to specialize on the selected subgoals. Experimental results demonstrate that our system is able to navigate a robot to the desired position effectively, and the HRL shows a much better learning performance than the standard RL in solving our navigation tasks.

Vulnerable Road User Detection using YOLO v3

Detection and classification of vulnerable road users (VRUs) is one of the most crucial blocks in vision based navigation systems used in Advanced Driver Assistance Systems. This paper seeks to evaluate the performance of object classification algorithm, You Only Look Once i.e. YOLO v3 algorithm for the purpose of detection of a major subclass of VRUs i.e. cyclists and pedestrians using the Tsinghua – Daimler dataset. The YOLO v3 algorithm used here requires less computational resources and hence promises a real time performance when compared to its predecessors. The model has been trained using the training images in the mentioned benchmark and have been tested for the test images available for the same. The average IoU for all the truth objects is calculated and the precision recall graph for different thresholds was plotted.

Vision Aided Navigation of a Quad-Rotor for Autonomous Wind-Farm Inspection

Abstract This work presents a vision based navigation system for an autonomous drone that is capable of recognizing and locating wind mills. WindMillNet, a Deep Neural Network created for this purpose was specially trained to recognize wind mills on camera images using transfer learning techniques. The drone powered by WindMillNet scans the horizon to find wind mills, and after perceiving a wind mill, navigates towards it, with the goal of performing its inspection. A hierarchical control system, implemented in the drone provides stability and control of its movements. Our framework was designed using a cyber-physical systems approach using high-level abstractions in modeling, communication, control and computation.

基于凸包的非合作目标视觉导航系统特征点选择方法

基于凸包的非合作目标视觉导航系统特征点选择方法

Intelligent vision-based navigation system for mobile robot: A technological review

Vision system is gradually becoming more important. As computing technology advances, it has been widely utilized in many industrial and service sectors. One of the critical applications for vision system is to navigate mobile robot safely. In order to do so, several technological elements are required. This article focuses on reviewing recent researches conducted on the intelligent vision-based navigation system for the mobile robot. These include the utilization of mobile robot in various sectors such as manufacturing, warehouse, agriculture, outdoor navigation and other service sectors. Multiple intelligent algorithms used in developing robot vision system were also reviewed.

A Vision-based navigation system for landing procedure

An autonomous vision-based landing system was designed and its performance is analysed and measured by an UAS. The system equipment is based on a single camera to determine its position and attitude with respect to a well-defined landing pattern. The developed procedure is based on photogrammetric Space Resection Solution, which provides the position and camera attitude reckoning starting from at least three, not aligned, reference control points whose image coordinates may be measured in the image camera frame. Five circular coloured targets were placed on a specific landing pattern, their 2D image frame coordinates was extracted through a particular algorithm. The aim of this work is to compute UAS precise position and attitude from single image, in order to have a good approach to landing field. This procedure can be used in addition or for replacement of GPS tracking and can be applied when the landing field is movable or located on a moving platform, the UAS will follow the landing pattern until the landing phase will be closed.

ROBUST POSITIONING OF DRONES FOR LAND USE MONITORING IN STRONG TERRAIN RELIEF USING VISION-BASED NAVIGATION

For land use monitoring, the main problems are robust positioning in urban canyons and strong terrain reliefs with the use of GPS system only. Indeed, satellite signal reflection and shielding in urban canyons and strong terrain relief results in problems with correct positioning. Using GNSS-RTK does not solve the problem completely because in some complex situations the whole satellite's system works incorrectly. We transform the weakness (urban canyons and strong terrain relief) to an advantage. It is a vision-based navigation using a map of the terrain relief. We investigate and demonstrate the effectiveness of this technology in Chinese region Xiaoshan. The accuracy of the vision-based navigation system corresponds to the expected for these conditions. . It was concluded that the maximum position error based on vision-based navigation is 20 m and the maximum angle Euler error based on vision-based navigation is 0.83 degree. In case of camera movement, the maximum position error based on vision-based navigation is 30m and the maximum Euler angle error based on vision-based navigation is 2.2 degrees.

A Vision-Based Navigation System for Perching Aircraft

This paper presents the investigation of the use of position-sensing diode (PSD) - a light source direction sensor - for designing a vision-based navigation system for a perching aircraft. Aircraft perching maneuvers mimic bird’s landing by climbing for touching down with low velocity or negligible impact. They are optimized to reduce their spatial requirements, like altitude gain or trajectory length. Due to disturbances and uncertainties, real-time perching is realized by tracking the optimal trajectories. As the performance of the controllers depends on the accuracy of estimated aircraft state, the use of PSD measurements as observations in the state estimation model is proposed to achieve precise landing. The performance and the suitability of this navigation system are investigated through numerical simulations. An optimal perching trajectory is computed by minimizing the trajectory length. Accelerations, angular-rates and PSD readings are determined from this trajectory and then added with experimentally obtained noise to create simulated sensor measurements. The initial state of the optimal perching trajectory is perturbed, and by assuming zero biases, extended Kalman filter is implemented for aircraft state estimation. It is shown that the errors between estimated and actual aircraft states reduce along the trajectory, validating the proposed navigation system.

A Non-Cooperative Satellite Feature Point Selection Method for Vision-Based Navigation System.

The number of feature points on the surface of a non-cooperative target satellite used for monocular vision-based relative navigation affects the onboard computational load. A feature point selection method called the quasi-optimal method is proposed to select a subset of feature points with a good geometric distribution. This method, with the assumption that all of the feature points are in a plane and have the same variance, is based on the fact that the scattered feature points can provide higher accuracy than that of them grouped together. The cost is defined as a function of the angle between two unit vectors from the projection center to feature points. The redundancy of a feature point is calculated by summing all costs associated with it. Firstly, the feature point with the most redundant information is removed. Then, redundancies are calculated again with the second feature point removed. The procedures above are repeated until the desired number of feature points is reached. Dilution of precision (DOP) represents the mapping relation between the observation variance and the estimated variance. In this paper, the DOP concept is used in a vision-based navigation system to verify the performance of the quasi-optimal method. Simulation results demonstrate the feasibility of calculating the relative position and attitude by using a subset of feature points with a good geometric distribution. It also shows that the feature points selected by the quasi-optimal method can provide a high accuracy with low computation time.

Online Observability-Constrained Motion Suggestion via Efficient Motion Primitive-Based Observability Analysis

An active perception methodology is proposed to locally predict the observability condition in a reasonable horizon and suggest an observability-constrained motion direction for the next step to ensure an accurate and consistent state estimation performance of vision-based navigation systems. The methodology leverages an efficient EOG-based observability analysis and a motion primitive-based path sampling technique to realize the local observability prediction with a real-time performance. The observability conditions of potential motion trajectories are evaluated, and an informed motion direction is selected to ensure the observability efficiency for the state estimation system. The proposed approach is specialized to a representative optimization-based monocular vision-based state estimation formulation and demonstrated through simulation and experiments to evaluate the ability of estimation degradation prediction and efficacy of motion direction suggestion.

UAV VISUAL AUTOLOCALIZATON BASED ON AUTOMATIC LANDMARK RECOGNITION

Abstract. Deploying an autonomous unmanned aerial vehicle in GPS-denied areas is a highly discussed problem in the scientific community. There are several approaches being developed, but the main strategies yet considered are computer vision based navigation systems. This work presents a new real-time computer-vision position estimator for UAV navigation. The estimator uses images captured during flight to recognize specific, well-known, landmarks in order to estimate the latitude and longitude of the aircraft. The method was tested in a simulated environment, using a dataset of real aerial images obtained in previous flights, with synchronized images, GPS and IMU data. The estimated position in each landmark recognition was compatible with the GPS data, stating that the developed method can be used as an alternative navigation system.

A voting method for stereo egomotion estimation

The development of vision-based navigation systems for mobile robotics applications in outdoor scenarios is a very challenging problem due to frequent changes in contrast and illumination, image bl...

Survey of Visual Servoing Control Schemes for Mobile Robot Navigation

In this paper, visual servoing control scheme for mobile robot navigation is discussed. The survey introduces a brief discussion of the subjects of the system under consideration. These subjects include mobile robot platform kinematics, vision systems, visual servoing, mapping, navigation, automatic control, and finally the avoidance techniques used to avoid obstacle during the navigation process. Different vision based navigation systems will be discussed and a comparative study will be given.