Monocular Visual Navigation Research Articles

A Visual Navigation Algorithm for UAV Based on Visual-Geography Optimization

The estimation of Unmanned Aerial Vehicle (UAV) poses using visual information is essential in Global Navigation Satellite System (GNSS)-denied environments. In this paper, we propose a UAV visual navigation algorithm based on visual-geography Bundle Adjustment (BA) to address the challenge of missing geolocation information in monocular visual navigation. This algorithm presents an effective approach to UAV navigation and positioning. Initially, Visual Odometry (VO) was employed for tracking the UAV’s motion and extracting keyframes. Subsequently, a geolocation method based on heterogeneous image matching was utilized to calculate the geographic pose of the UAV. Additionally, we introduce a tightly coupled information fusion method based on visual-geography optimization, which provides a geographic initializer and enables real-time estimation of the UAV’s geographical pose. Finally, the algorithm dynamically adjusts the weight of geographic information to improve optimization accuracy. The proposed method is extensively evaluated in both simulated and real-world environments, and the results demonstrate that our proposed approach can accurately and in real-time estimate the geographic pose of the UAV in a GNSS-denied environment. Specifically, our proposed approach achieves a root-mean-square error (RMSE) and mean positioning accuracy of less than 13 m.

Monocular Visual Navigation Algorithm for Nursing Robots via Deep Learning Oriented to Dynamic Object Goal

Monocular Visual Navigation Algorithm for Nursing Robots via Deep Learning Oriented to Dynamic Object Goal

基于PID-模糊控制的单目视觉农田机器人视觉导航系统设计

This study proposes a monocular vision navigation control system based on PID-fuzzy control, which travels along the edge of the path. It collects path image information through monocular vision, identifies the path edge through image processing to determine the preview point, and uses a combination of PID and fuzzy control to design a controller to track the preview point for path navigation. Firstly, coordinate calibration and conversion were performed on the monocular camera, achieving coordinate conversion from the image coordinate system to the world coordinate system. The accuracy of the calibration results was verified through experiments. According to the navigation strategy of driving along the edge of the path, the world coordinate equation of the path edge is obtained through image processing technology, and the preview point tracked by the navigation system is determined. The navigation parameters are determined based on the position of the preview point. The PID fuzzy controller system designed in this study can switch different control methods based on the position of the preview point. Finally, an experimental verification was conducted on the monocular visual navigation system of the control system. The verification results showed that the average error of the navigation control system in tracking the path when driving in a straight line was 0.039 m, the average error when turning left was 0.079 m, and the average error when turning right was 0.121 m. The error range can meet the basic requirements of agricultural robot farmland operations. Research has shown that the navigation strategy based on PID-fuzzy joint controller to track the preview point along the path edge has a good effect on the visual navigation control system of agricultural robots. This study provides important reference value for the research and development of monocular visual navigation systems of agricultural robots.

Brain computer interface system based on monocular vision and motor imagery for UAV indoor space target searching

This paper puts forward a brain computer interface (BCI) system that integrates the monocular vision navigation and decision subsystems to realize the indoor three-dimensional (3D) space target searching with a low speed Unmanned Aerial Vehicle (UAV). The monocular vision navigation subsystem provides 3D space feasible flight directions for the decision subsystem. It utilizes the Scale Invariant Feature Transform (SIFT) algorithm and Brute-Force (BF) algorithm to extract the key points and match the extracted feature points. Obstacles and their positions are estimated by comparing the changes in the size of key points and the ratio of the size of the “convex points” area of the interest targets. The decision subsystem collects four kinds of motor imaginary (MI) tasks (left/right-hand, feet and tongue) electroencephalogram (EEG) signals from 15 electrodes firstly. Then, it applies two fifth-order Butterworth Band-Pass Filter (BPF) to preprocess the raw MI tasks EEG signals. Next, it adopts the Common Spatial Pattern (CSP) to spatially filter the preprocess EEG signals. To implement the quaternary classification,the decision subsystem utilizes the single convolutional layer Convolutional Neural Network (CNN) to realize the MI tasks EEG signals feature extraction, classification and final feasible flight direction decision finally. During this process, the spectrograms of the spatially filtered signals are given as the input to CNN. The actual indoor 3D space target searching experiment verifies that this BCI system has good adaptability and control stability.

Intelligent Multi-drive Inspection and Positioning Method for Water Environment of Cable Pipe Corridor Based on Artificial Intelligence

The running environment of underground power cables is hidden, and it is difficult to identify the cables in the parallel cable trench, which is not conducive to the rapid troubleshooting and accurate positioning of faults. However, the traditional manual inspection method has some defects, such as difficulty in finding, recording, low efficiency and inconvenient re-use of inspection information. Based on the above background, the purpose of this study is to explore the intelligent multi drive inspection and positioning method for water environment of cable pipe gallery based on artificial intelligence. Based on the analysis of the existing cable tunnel water environment inspection technology, this paper introduces the basic theory of machine vision and artificial intelligence to analyze the requirements of the water environment inspection and positioning method of the cable pipe gallery. Firstly, an intelligent multi drive inspection and positioning overall structure is designed. Secondly, aiming at the problem of unattended cable tunnel water environment inspection and navigation positioning, A monocular vision navigation scheme based on guide line is developed, which includes two parts: walking along the line and detecting the fixed point. Through the intelligent multi drive inspection and positioning method, the work efficiency of the water environment inspection of the cable tunnel is greatly improved. The reference error of the identification method is less than 1.8%. At the same time, the inspection data can be completely saved, which is conducive to the early warning of cable equipment defects and the analysis of the development trend, and the reliable and stable operation level of the field equipment and power grid of the cable tunnel is improved.

Virtual testbed for monocular visual navigation of small unmanned aircraft systems

Monocular visual navigation methods have seen significant advances in the last decade, recently producing several real-time solutions for autonomously navigating small unmanned aircraft systems without relying on the Global Positioning System (GPS). This is critical for military operations that may involve environments where GPS signals are degraded or denied. However, testing and comparing visual navigation algorithms remains a challenge since visual data is expensive to gather. Conducting flight tests in a virtual environment is an attractive solution prior to committing to outdoor testing. This work presents a virtual testbed for conducting simulated flight tests over real-world terrain and analyzing the real-time performance of visual navigation algorithms at 31 Hz. This tool was created to ultimately find a visual odometry algorithm appropriate for further GPS-denied navigation research on fixed-wing aircraft, even though all of the algorithms were designed for other modalities. This testbed was used to evaluate three current state-of-the-art, open-source monocular visual odometry algorithms on a fixed-wing platform: Direct Sparse Odometry, Semi-Direct Visual Odometry, and ORB-SLAM2 (with loop closures disabled).

RBF-Based Monocular Vision Navigation for Small Vehicles in Narrow Space below Maize Canopy

Maize is one of the major food crops in China. Traditionally, field operations are done by manual labor, where the farmers are threatened by the harsh environment and pesticides. On the other hand, it is difficult for large machinery to maneuver in the field due to limited space, particularly in the middle and late growth stage of maize. Unmanned, compact agricultural machines, therefore, are ideal for such field work. This paper describes a method of monocular visual recognition to navigate small vehicles between narrow crop rows. Edge detection and noise elimination were used for image segmentation to extract the stalks in the image. The stalk coordinates define passable boundaries, and a simplified radial basis function (RBF)-based algorithm was adapted for path planning to improve the fault tolerance of stalk coordinate extraction. The average image processing time, including network latency, is 220 ms. The average time consumption for path planning is 30 ms. The fast processing ensures a top speed of 2 m/s for our prototype vehicle. When operating at the normal speed (0.7 m/s), the rate of collision with stalks is under 6.4%. Additional simulations and field tests further proved the feasibility and fault tolerance of our method.

Time domain optimisation for monocular visual navigation: moving horizon approach

Estimating ego motion of monocular visual system from the input image sequence is a critical problem in computer vision. This paper considers this problem from the view of time domain optimisation. At first, based on the frame to frame matches, the traditional monocular visual navigation algorithm is introduced, and this process is also seen as a two-view method. On the basis of the three-view matches, the trifocal tensor based monocular navigation algorithm is described. In addition, this paper proposes a novel moving horizon estimation (MHE) based algorithm to tackle the pose estimation problem. In this algorithm, the epipolar constraints and the closed loop constraints of position along with consecutive time steps are all involved in the global optimisation model. In accordance with this optimisation function, the MHE is introduced to achieve the tradeoff between computation costs and estimation accuracy. Because of the inherent robustness of the referred moving horizon estimator, it is becoming more popular in various applications. Based on the general epipolar constraints, trifocal tensor constraints and step crossed optimisation function during the whole moving window, the corresponding three referred pose estimation algorithms are all implemented comparatively at the end of this paper. The rotation and translation experiments are implemented to validate the improvements of the time domain optimisation methods.

Robust and accurate monocular visual navigation combining IMU for a quadrotor

In this paper, we present a multi-sensor fusion based monocular visual navigation system for a quadrotor with limited payload, power and computational resources. Our system is equipped with an inertial measurement unit (IMU), a sonar and a monocular down-looking camera. It is able to work well in GPS-denied and markerless environments. Different from most of the keyframe-based visual navigation systems, our system uses the information from both keyframes and keypoints in each frame. The GPU-based speeded up robust feature (SURF) is employed for feature detection and feature matching. Based on the flight characteristics of quadrotor, we propose a refined preliminary motion estimation algorithm combining IMU data. A multi-level judgment rule is then presented which is beneficial to hovering conditions and reduces the error accumulation effectively. By using the sonar sensor, the metric scale estimation problem has been solved. We also present the novel IMU+3P (IMU with three point correspondences) algorithm for accurate pose estimation. This algorithm transforms the 6-DOF pose estimation problem into a 4-DOF problem and can obtain more accurate results with less computation time. We perform the experiments of monocular visual navigation system in real indoor and outdoor environments. The results demonstrate that the monocular visual navigation system performing in real-time has robust and accurate navigation results of the quadrotor.

A New Feature Points Reconstruction Method in Spacecraft Vision Navigation

The important applications of monocular vision navigation in aerospace are spacecraft ground calibration tests and spacecraft relative navigation. Regardless of the attitude calibration for ground turntable or the relative navigation between two spacecraft, it usually requires four noncollinear feature points to achieve attitude estimation. In this paper, a vision navigation system based on the least feature points is designed to deal with fault or unidentifiable feature points. An iterative algorithm based on the feature point reconstruction is proposed for the system. Simulation results show that the attitude calculation of the designed vision navigation system could converge quickly, which improves the robustness of the vision navigation of spacecraft.

Camera Self-Calibration in the AUV Monocular Vision Navigation and Positioning

camera calibration is essential to obtain 3D information from 2D image, for underwater camera self-calibration, because the linear model can not accurately describe the imaging geometry of real cameras, the underwater camera nonlinear model and its calibration method are studied. According to the relationship between underwater and air in the camera focal length, principal point and no vertical factors, the underwater camera imaging geometry is modeled. Underwater camera nonlinear imaging geometry is modeled with the radial distortion. The constraint equation of the nonlinear model parameters are set up with linear imaging model and fundamental matrix of the matching points, which realize the camera self-calibration of nonlinear parameters. For the linear parameters of the camera, with the principal point and no vertical factor known, the self calibration of the focal length can be realized with two images, according to the Kruppa equation,. Simulation and real image experiments show that the method given is feasible, and has certain practical value. DOI: http://dx.doi.org/10.11591/telkomnika.v11i12.3726 Full Text: PDF

Incremental parameterized modeling and its application in monocular visual navigation

Incremental parameterized modeling and its application in monocular visual navigation

Research on Visual Navigation and Remote Monitoring Technology of Agricultural Robot

Abstract To solve the problems of instability of agricultural robot when avoiding obstacles, a kind of navigation method which combined monocular visual navigation technology and remote monitoring technology was proposed: using the centerline method to extract the navigation path to achieve visual navigation; the monitoring center received two-way real-time image signals of the agricultural robot, when the agricultural robot met the obstacle or other situations, the remote monitoring center would start the alarm, then the operators could send control signals through the monitoring software to implement manual intervention. The experiment showed that the system improved the reliability of the navigation.

Monocular Vision Navigation and Control of Mobile Robot

In this paper, we implement a monocular vision navigation system on a mobile robot which can travel in known outdoor environment. The system uses 5 different sensors on the robot which are monocular color camera, laser radar, sonar, gyroscopes and optical encoder. And the robot can cruise independently in known continuous structured and unstructured road environment mainly through the vision navigation module. There are road recognition module, signs recognition module, stop line recognition module, zebra crossing recognition module, obstacle sensing module, control module and communication module. The system utilizes multi-threading technology to collaborate different modules in the same control cycle and automatic state machine to switch different control modes. Experimental results show that the robot can accomplish autonomous cruise task on the predetermined route and operate stably when it is in a good lighting conditions.