Monocular Camera Research Articles

A bio-inspired looming detection for stable landing in unmanned aerial vehicles**This research was funded by the National Natural Science Foundation ofchina, grant number 62206065, and the Bagui Scholar Program of Guangxi.

Flying insects, such as flies and bees, have evolved the capability to rely solely on visual cues for smooth and secure landings on various surfaces. In the process of carrying out tasks, micro unmanned aerial vehicles (UAVs) may encounter various emergencies, and it is necessary to land safely in complex and unpredictable ground environments, especially when altitude information is not accurately obtained, which undoubtedly poses a significant challenge. Our study draws on the remarkable response mechanism of the Lobula Giant Movement Detector to looming scenarios to develop a novel UAV landing strategy. The proposed strategy does not require distance estimation, making it particularly suitable for payload-constrained micro aerial vehicles. Through a series of experiments, this strategy has proven to effectively achieve stable and high-performance landings in unknown and complex environments using only a monocular camera. Furthermore, a novel mechanism to trigger the final landing phase has been introduced, further ensuring the safe and stable touchdown of the drone.

An Efficient Dense Reconstruction Algorithm from LiDAR and Monocular Camera

Dense reconstruction have been studied for decades in the fields of computer vision and robotics, in which LiDAR and camera are widely used. However, vision-based methods are sensitive to illumination variation and lack direct depth, and LiDAR-based methods are limited by sparse LiDAR measurement and lacking color and texture information. In this paper, we propose a novel 3D reconstruction algorithm based on LiDAR and a monocular camera, which realizes dense reconstruction. In the algorithm, a LiDAR odometry is used to get accurate poses and poses calculated by the odometry module are used in the calculation of depth maps and fusion of depth maps, and then mesh and texture mapping are implemented. In addition, a semantic segmentation network and a depth completion network are used to obtain dense and accurate depth maps. The concept of symmetry is utilized to generate 3D models of objects or scenes; that is, the reconstruction and camera imaging of these objects or scenes are symmetrical. Experimental results on public dataset show that the proposed algorithm achieves higher accuracy, efficiency and completeness than existing methods.

Analysis of modeling nasal narrow space based on visual slam

Abstract. Now, surgeries are becoming more stable, safe, efficient and low-cost. During the surgical treatment of nasal diseases, surgical robotic robotics can help operate accurately and reduce the discomfort after surgery. However, due to the internal space of the nasal cavity being relatively narrow, it is difficult for the nasal surgical robot to contain multiple vision sensors and the monocular camera could not get information about the depth of the 3D objects in the scene, so the existing surgical robots cannot accomplish the three-dimensional modeling about the internal space of nasal cavity well. In practice, doctors still have to analyze pictures from the robotic, which may decrease the efficiency of the surgery and increase the risk to patients. This article designed a SLAM algorithm framework based on a depth estimation network, it can simulate the internal structure of the nasal cavity more accurately through pictures, which come from monocular endoscopic on surgical robotic. The insights gained in this study verify that the method of image segmentation can also make the depth representation of the nasal internal space more accurate and this method may help robots realize their self-position in the narrow area of the nasal cavity, which lays the foundations for the development of fully autonomous surgical robots.

Passive Stylus Tracking: A Systematic Literature Review

Passive stylus systems offer a simple and cost-effective solution for digital input, compatible with a wide range of surfaces and devices. This study reviews the domain of passive stylus tracking on passive surfaces, a topic previously underexplored in existing literature. We answer four key research questions: what type of systems exist in this domain, what methods do they use for tracking styli, how accurate are they, and what are their limitations? A systematic literature review resulted in 24 papers describing passive stylus systems. Their methods primarily fall into four categories: monocular cameras with image processing, multiple camera systems with image processing, machine learning systems using high-speed cameras or motion capture hardware, and radio frequency signal-based systems with signal processing. We found the system with the highest accuracy used a single monocular camera. In many systems, markers such as retroreflective spheres, tape, or fiducial markers were used to enhance the feature matching. We have also found stagnation and in some cases, regression in the precision and reliability of these systems over time. The limitations in these systems include the lack of varied stylus form factor support, the restriction to specific camera positions and angles, and the requirement of expensive hardware. Given these findings, we discuss the important characteristics and features of passive stylus systems and propose ways forward in this field.

Cross-Spectral Navigation with Sensor Handover for Enhanced Proximity Operations with Uncooperative Space Objects

Close-proximity operations play a crucial role in emerging mission concepts, such as Active Debris Removal or small celestial bodies exploration. When approaching a non-cooperative target, the increased risk of collisions and reduced reliance on ground intervention necessitate autonomous on-board relative pose (position and attitude) estimation. Although navigation strategies relying on monocular cameras which operate in the visible (VIS) spectrum have been extensively studied and tested in flight for navigation applications, their accuracy is heavily related to the target’s illumination conditions, thus limiting their applicability range. The novelty of the paper is the introduction of a thermal-infrared (TIR) camera to complement the VIS one to mitigate the aforementioned issues. The primary goal of this work is to evaluate the enhancement in navigation accuracy and robustness by performing VIS-TIR data fusion within an Extended Kalman Filter (EKF) and to assess the performance of such navigation strategy in challenging illumination scenarios. The proposed navigation architecture is tightly coupled, leveraging correspondences between a known uncooperative target and feature points extracted from multispectral images. Furthermore, handover from one camera to the other is introduced to enable seamlessly operations across both spectra while prioritizing the most significant measurement sources. The pipeline is tested on Tango spacecraft synthetically generated VIS and TIR images. A performance assessment is carried out through numerical simulations considering different illumination conditions. Our results demonstrate that a combined VIS-TIR navigation strategy effectively enhances operational robustness and flexibility compared to traditional VIS-only navigation chains.

Perspective Transform-based Depth Estimation of Monocular Camera for Electrocution Threat Determination of Construction Machinery

Abstract. This paper discusses the application of perspective transform-based monocular camera depth estimation method in monitoring the safety distance between construction machinery and power lines. The traditional distance measurement relies on manual operation, which is inefficient and not precise enough, while the monocular depth estimation method based on deep learning can improve the accuracy, but faces high equipment cost and relies on a large amount of training data. In contrast, the depth estimation method based on perspective transformation proposed in this paper utilizes mathematical and physical principles to establish a mathematical model between the pixel distances of an image and the actual scene distances through perspective projection theory, which simplifies the hardware requirements and reduces the data dependence and improves the computational efficiency and applicability. Through experimental validation under different construction scenes and exposure conditions, the method demonstrates high accuracy and robustness, proving its practicality in construction safety management. This research provides a new technical direction and practical application possibility in the field of intelligent monitoring and 3D reconstruction.

Hybrid Mobile Robot Path Planning Using Safe JBS-A*B Algorithm and Improved DWA Based on Monocular Camera

This paper addresses the formidable challenge of enabling autonomous navigation in Mobile Robots (MRs), focusing on the development of advanced path planning strategies. Despite their pivotal role in diverse applications, they face challenges in dynamic settings due to limitation in existing Global Path Planning (GPP) and Local Path Planning (LPP) techniques. In response to this, we propose an innovative hybrid path planning approach that enhances the A* algorithm with a risk-aware heuristic function and integrates the Jump Point Search (JPS) technique for route optimization. Additionally, B-spline smoothing is employed for perceptually global trajectory refinement. Our approach also includes an innovative improvement to the Dynamic Window Approach (DWA) to align with the proposed enhanced A* algorithm for effective local navigation. Acknowledging the importance of high-quality input in path planning, we present substantial improvements to the IRDC-Net, a monocular-image semantic-segmentation model that we studied previously. Novel improvements include the integration of quantization and the Adam optimizer, along with the implementation of the Balanced Cross-Entropy loss function. These enhancements not only elevate the output quality of IRDC-Net but also reduce the model’s training parameters. The experimental results demonstrate the performance and viability of the proposed algorithm. Ultimately, the hybrid MR’s path planning algorithm exhibits proficiency across various tasks, particularly in addressing the challenge of evading moving obstacles to ensure the robot’s safety while adhering to the global path.

A Scaled Monocular 3D Reconstruction Based on Structure from Motion and Multi-View Stereo

Three-dimensional digital modeling at actual scales is essential for digitally preserving cultural relics. While 3D reconstruction using a monocular camera offers a cost-effective solution, the lack of scale information in the resulting models limits their suitability for geometric measurements. Objects with monotonous textures, such as batteries, pose additional challenges due to insufficient feature points, increasing positional uncertainty. This article proposes a method incorporating point and line features to address the scale ambiguity in multi-view 3D reconstruction using monocular cameras. By pre-measuring the lengths of multiple sets of real line segments, building a lookup table, and associating the line features in different images, the table was input into the improved reconstruction algorithm to further optimize the scale information. Experimental results on real datasets showed that the proposed method outperformed the COLMAP method by 70.82% in reconstruction accuracy, with a scale recovery reaching millimeter-level accuracy. This method is highly generalizable, cost-effective, and supports lightweight computation, making it suitable for real-time operation on a CPU.

Understanding amorphous gorge scenes based on the projection of spatial textures

Geographic information and topographical understanding are crucial for effective field environmental monitoring and autonomous navigation. Unfortunately, implementing this knowledge is often difficult due to unsolved problems such as understanding disorganized and unstructured mountainous regions. Traditional three-dimensional (3D) scene understanding methods from 3D point clouds or fused data are energy-consuming and unsuitable for a resource-constrained unmanned aerial vehicle (UAV) with limited computation, memory, and energy. This study proposes a methodology to understand unstructured gorges and reconstruct them in 3D environments from a low-cost monocular camera. The new approach assigns mountain texture projections to clusters and reshapes surfaces of unstructured mountains to follow geometric constraints regarding position and orientation. The relative relationships between surfaces and vanishing points are then analyzed to understand and reconstruct amorphous gorge scenes. This method is practical and efficient for navigating amorphous gorges based on understanding unstructured mountains. Fortunately, this method requires no prior training and is robust to color and illumination. We compared the percentage of incorrectly classified pixels with the ground truth. Experimental results have demonstrated that our approach can successfully understand amorphous gorges, meeting the requirements for autonomous flying in mountainous environments.

Monocular visual detection of coal flow rate in scraper conveyor based on template matching background differencing

Abstract The monitoring of coal flow is a crucial aspect of the intelligent regulation and control of comprehensive mining equipment. In recent years, machine vision technology has become a mainstream method for quickly and efficiently extracting coal flow information. However, the majority of research in this field has focused on belt conveyors, with relatively limited investigation into the use of this technology with scraper conveyors. In order to address the need for monitoring coal flow in scraper conveyors, a monocular visual detection method of coal flow rates based on template matching-background differencing is proposed. First, the region of interet in the images captured using a monocular camera mounted at a specific location is quickly identified using an enhanced template matching method. Second, the image motion region is segmented using interframe and background differencing. Finally, the coal flow rate is calculated on the basis of the number of pixel points in the segmented image. Experimental verification is performed using scraper conveyor test bench and real underground data. The results demonstrate that the proposed coal flow detection method is capable of achieving real-time detection of coal flow in scraper conveyor and provides a theoretical basis for the monitoring of coal flow of the scraper conveyor.

Feline eye-inspired artificial vision for enhanced camouflage breaking under diverse light conditions.

Biologically inspired artificial vision research has led to innovative robotic vision systems with low optical aberration, wide field of view, and compact form factor. However, challenges persist in object detection and recognition against complex backgrounds and varied lighting. Inspired by the feline eye, which features a vertically elongated pupil and tapetum lucidum, this study introduces an artificial vision system designed for superior object detection and recognition in a monocular framework. Using a slit-like elliptical aperture and a patterned metal reflector beneath a hemispherical silicon photodiode array, the system reduces excessive light and enhances photosensitivity. This design achieves clear focus under bright light and enhanced sensitivity in dim conditions. Theoretical and experimental analyses demonstrate the system's ability to filter redundant information and detect camouflaged objects in diverse lighting, representing a substantial advancement in monocular camera technology and the potential of biomimicry in optical innovations.

Obstacle feature point detection method for real-time processing by monocular camera and line laser

In this study, we propose a method for detecting feature points of obstacles by real-time processing using a monocular camera and a laser. Specifically, we propose an algorithm that detects laser beams emitted on obstacles by binarization, transforms pixel coordinates into distance using triangulation by a laser beam that is obliquely incident on the ground, and estimates the placement angle of the obstacle in real-time processing. No method has been devised to detect the placement angle of obstacles using a monocular camera and a laser. Furthermore, the proposed method does not calculate the linear distance from the camera, but from the position where the camera is moved horizontally parallel to the obstacle in front of the camera, thus detecting the placement angle independently of the placement position of the obstacle. As a result, it was confirmed that the placement angles of obstacles could be detected with a maximum error of 6∘\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$6^{\\circ }$$\\end{document} in six placement positions. Therefore, we succeeded in automatically detecting the placement angle of obstacles in real-time, independent of the illumination of the measurement environment, the reflectance of the obstacle.

An efficient and automatic method based on monocular camera and GNSS for collecting and updating geographical coordinates of mileage pile in highway digital twin map

Abstract The high-precision positioning of mileage piles on a digital map ensures accurate data for high-speed event releases, toll audits, and road condition monitoring. This paper introduces an efficient and automatic system for positioning highway mileage piles. The self-developed data acquisition system collects mileage pile images and road trajectory. Considering the limited variety of mileage piles, a Simplified-YOLOv5m (S-YOLOv5m) is proposed. Then a high-precision character detection network of S-YOLOv5m is proposed to enhance feature extraction and improve the accuracy of character detection on mileage piles. Thirdly, the end-to-end monocular distance measurement combines the target detection with the distance estimation, enabling simultaneous object detection and distance measurement. Fourthly, by combining with the geographical coordinates of the acquisition point, the direct solution to geodetic problems is applied to calculate the spatial coordinates of the mileage piles. Finally, the missing mileage piles are compensated for using the road curve and equal distance interpolation. The complete mileage piles and geographical information list of the inspection trajectory are output. Through a series of verification tests, the average positioning error of the mileage pile system is 1.265 m. The contribution of the automatic positioning system of mileage piles is to construct the relationship between mileage piles and geographical coordinates, match mileage piles with digital maps to realize the full-featured, real-scene, and high-dynamic management of road attributes.

YLS-SLAM: a real-time dynamic visual SLAM based on semantic segmentation

Purpose Traditional visual simultaneous localization and mapping (SLAM) systems are primarily based on the assumption that the environment is static, which makes them struggle with the interference caused by dynamic objects in complex industrial production environments. This paper aims to improve the stability of visual SLAM in complex dynamic environments through semantic segmentation and its optimization. Design/methodology/approach This paper proposes a real-time visual SLAM system for complex dynamic environments based on YOLOv5s semantic segmentation, named YLS-SLAM. The system combines semantic segmentation results and the boundary semantic enhancement algorithm. By recognizing and completing the semantic masks of dynamic objects from coarse to fine, it effectively eliminates the interference of dynamic feature points on the pose estimation and enhances the retention and extraction of prominent features in the background, thereby achieving stable operation of the system in complex dynamic environments. Findings Experiments on the Technische Universität München and Bonn data sets show that, under monocular and Red, Green, Blue - Depth modes, the localization accuracy of YLS-SLAM is significantly better than existing advanced dynamic SLAM methods, effectively improving the robustness of visual SLAM. Additionally, the authors also conducted tests using a monocular camera in a real industrial production environment, successfully validating its effectiveness and application potential in complex dynamic environment. Originality/value This paper combines semantic segmentation algorithms with boundary semantic enhancement algorithms to effectively achieve precise removal of dynamic objects and their edges, while ensuring the system's real-time performance, offering significant application value.

A Target Detection Algorithm Based on Fusing Radar with a Camera in the Presence of a Fluctuating Signal Intensity

Radar point clouds will experience variations in density, which may cause incorrect alerts during clustering. In turn, it will diminish the precision of the decision-level fusion method. To address this problem, a target detection algorithm based on fusing radar with a camera in the presence of a fluctuating signal intensity is proposed in this paper. It introduces a snow ablation optimizer (SAO) for solving the optimal parameters of the density-based spatial clustering of applications with noise (DBSCAN). Subsequently, the enhanced DBSCAN clusters radar point clouds, and the valid clusters are fused with monocular camera targets. The experimental results indicate that the suggested fusion method can attain a Balance-score ranging from 0.97 to 0.99, performing outstandingly in preventing missed detections and false alarms. Additionally, the fluctuation range of the Balance-score is within 0.02, indicating the algorithm has an excellent robustness.

An improved DFD method for three-dimensional displacement measurement of vision-based tactile sensor

Currently, traditional tactile sensors based on the principles of capacitance or piezoelectricity have complex structures and difficulty in obtaining tactile information. A vision-based tactile sensor is introduced which can realize visual measurement of three-dimensional displacement in this paper. The vision-based tactile sensor is mainly composed of an elastomer embedded with marker point array, a transparent acrylic plate, 8 LED lights and a micro monocular camera. The elastomer deforms when the tactile sensor contacts an object, and the micro monocular camera is used to capture the elastomer deformation and transmit it to the computer in the form of image, and then the three-dimensional displacement information is obtained by processing the image. In order to more accurately recover the missing dimensional information in the three-dimensional displacement detection of monocular camera, an improved DFD (Depth from Defocus) method based on finite element theory is proposed in this paper. It is verified by experiments that the improved DFD method proposed in this paper can measure the three-dimensional displacement information more accurately compared with the DFD method. In addition, an experiment is conducted to prove the robustness of the improved DFD method on the robotic gripper. The experimental results demonstrate that the three-dimensional displacement measurement method proposed in this paper can provide technical support for the design and development of vision-based tactile sensors.

Hybrid Visual Odometry Algorithm Using a Downward-Facing Monocular Camera

The increasing interest in developing robots capable of navigating autonomously has led to the necessity of developing robust methods that enable these robots to operate in challenging and dynamic environments. Visual odometry (VO) has emerged in this context as a key technique, offering the possibility of estimating the position of a robot using sequences of onboard cameras. In this paper, a VO algorithm is proposed that achieves sub-pixel precision by combining optical flow and direct methods. This approach uses only a downward-facing, monocular camera, eliminating the need for additional sensors. The experimental results demonstrate the robustness of the developed method across various surfaces, achieving minimal drift errors in calculation.

High-Accuracy, Vision-Based Attitude Estimation System for Air-Bearing Spacecraft Simulators

Air-bearing platforms for simulating the attitude dynamics of satellites require highly accurate ground truth systems. Commercial off-the-shelf motion-capture systems are used for this scope, although they are complex and expensive. This paper shows a novel and versatile method to estimate the attitude of rotational air-bearing platforms using a monocular camera and sets of fiducial LED markers. The work proposes a geometry-based, iterative algorithm that is significantly more accurate than other literature methods that involve solving the perspective-n-point problem. Additionally, autocalibration procedures to perform a preliminary estimation of the system parameters are shown. The developed methodology is deployed onto a Raspberry Pi 4 microcomputer and tested with an array of LEDs soldered on a custom printed circuit board. Data obtained with this setup are compared against simulations of the system to determine and validate its performance. The hardware setup is also validated against independent inclinometric and gyroscope sensor measurements. Simulation and test results show expected 1-sigma accuracies of 12 arcs and 37 arcs for about- and cross-boresight rotations of the platform, respectively, and average latency times of 6 ms.

Development of an Automatic Feature Point Classification Method for Three-Dimensional Mapping Around Slewing and Derricking Cranes

Crane automation requires a three-dimensional (3D) map around cranes that should be reconstructed and updated quickly.In this study, a high-precision classification method was developed to distinguish stationary objects from moving objects in moving images captured by a monocular camera to stabilize 3D reconstruction. To develop the method, a moving image was captured while the crane was slewed with a monocular camera mounted vertically downward at the tip of the crane. The boom length and angle data were output from a control device, a controller area network. For efficient development, a simulator that imitated the environment of an actual machine was developed and used. The proposed method uses optical flow to track feature points. The classification was performed successfully, independent of derricking motion. Consequently, the proposed method contributes to stable 3D mapping around cranes in construction sites.

Visual Odometry in GPS-Denied Zones for Fixed-Wing Unmanned Aerial Vehicle with Reduced Accumulative Error Based on Satellite Imagery

In this paper, we present a method for estimating GPS coordinates from visual information captured by a monocular camera mounted on a fixed-wing tactical Unmanned Aerial Vehicle at high altitudes (up to 3000 m) in GPS-denied zones. The main challenge in visual odometry using aerial images is the computation of the scale due to irregularities in the elevation of the terrain. That is, it is not possible to accurately convert from pixels in the image to meters in space, and the error accumulates. The contribution of this work is a reduction in the accumulated error by comparing the images from the camera with satellite images without requiring the dynamic model of the vehicle. The algorithm has been tested in real-world flight experiments at altitudes above 1000 m and in missions over 17 km. It has been proven that the algorithm prevents an increase in the accumulated error.