Inverse Depth Research Articles

DVDS: A deep visual dynamic slam system

Simultaneous localization and mapping (SLAM) utilizing visual sensors represent an extensively investigated research area, holding significant potential for advancements in robotics and autonomous vehicular systems. Recently, dense SLAM systems underpinned by learning-based methodologies have showcased superior accuracy and robustness compared to conventional techniques. Nevertheless, contemporary learning-based SLAM systems exhibit notable discrepancies in pose estimation, particularly within dynamic environments. In addition, the constrained receptive field of convolutional features in these methods impedes their efficacy when confronted with homogeneous, texture-less images, rendering them vulnerable to noise perturbations. We develop a novel deep visual dynamic slam (DVDS) system that exploits solely static pixels within images to retrieve the camera poses. Specifically, we formulate a dynamic object exclusion mechanism that excises dynamic constituents within the scene before the optical flow computation, thus optimizing the precision of the estimation. In addition, we unveil an efficient dispersive transformer (DisFormer) that facilitates per-pixel features in assimilating long-range information from surrounding features, culminating in constructing more precise 4D correlation volumes. Building on the DisFormer, we suggest a Disformer-based gated recurrent unit (GRU) to generate a refined flow field coupled with a confidence map, which is subsequently employed by the dense bundle adjustment layer to iteratively rectify the residuals of inverse depths and associated camera poses. The global receptive field provided by the DisFormer promotes information integration from a wider contextual window, thus improving the robustness of our SLAM system. Comprehensive experiments underscore that our proposed DVDS system manifests superior efficacy compared with state-of-the-art works across both static and dynamic scenes.

H-SLAM: Hybrid direct–indirect visual SLAM

The recent success of hybrid methods in monocular odometry has led to many attempts to generalize the performance gains to hybrid monocular SLAM. However, most attempts fall short in several respects, with the most prominent issue being the need for two different map representations (local and global maps), with each requiring different, computationally expensive, and often redundant processes to maintain. Moreover, these maps tend to drift with respect to each other, resulting in contradicting pose and scene estimates, and leading to catastrophic failure. In this paper, we propose a novel approach that makes use of descriptor sharing to generate a single inverse depth scene representation. This representation can be used locally, queried globally to perform loop closure, and has the ability to re-activate previously observed map points after redundant points are marginalized from the local map, eliminating the need for separate map maintenance processes. The maps generated by our method exhibit no drift between each other, and can be computed at a fraction of the computational cost and memory footprint required by other monocular SLAM systems. Despite the reduced resource requirements, the proposed approach maintains its robustness and accuracy, delivering performance comparable to state-of-the-art SLAM methods (e.g., LDSO, ORB-SLAM3) on the majority of sequences from well-known datasets like EuRoC, KITTI, and TUM VI. The source code is available at: https://github.com/AUBVRL/fslam_ros_docker.

Spatial analysis of leachate penetration at Lemna dumpsite, Calabar: Implications for sustainable waste management in Cross River State

This study rigorously investigated the spatial analysis of leachate penetration at Lemna dumpsite, located in Calabar, Cross River State, Nigeria. Purposeful soil sampling, performed at specific intervals (5 m, 25 m, and 50 m) along the Electrical Resistivity profile line within the dumpsite, was augmented by water sample collection from five boreholes near Lemna dumpsite. Utilizing Electrical Resistivity Tomography (ERT) and Vertical Electric Sounding (VES) survey techniques, resistivity data were systematically gathered to comprehensively analyze the Leachate Penetration in the Lemna dumpsite. Laboratory analysis of soil and borehole water quality focused on Benzene, Toluene, Ethylbenzene, and Xylene (BTEX), with paired sample t-tests applied for statistical scrutiny. Analyzing the ERT and VES data employed sophisticated techniques embedded in Resistivity Two Dimension Invasion software and Advanced Geosciences Incorporation Earth Imager software. Substantial disparities (p < 0.05) emerged in the paired sample t-tests for BTEX in soil compared to National Environmental Standard Regulation and Enforcement Agency (NESREA) limits. Similarly, BTEX in borehole water displayed significant differences (p < 0.05) when compared to World Health Organization (WHO) standards, raising alarming concerns about the safety and portability of groundwater in the area. The examination of dumpsite leachate penetration revealed a resistivity anomaly of 8.01 Ωm and an inverse depth of 12.4 m, underscoring profound environmental implications and necessitating immediate remediation efforts. Additionally, Vulnerability and Aquifer Protective Capacity Index (VES) results, with a rating of <0.1, indicated severely compromised aquifer protective capacity, emphasizing the vulnerability of groundwater resources to further contamination. Our study advocates for strategic management, remediation, and monitoring measures to prevent contamination and safeguard water quality in the region.

Extrinsic calibration for large FOV based on inverse depth parameterized bundle adjustment

To overcome the reliance on high-precision calibration plates in camera calibration, this paper proposes an extrinsic calibration method suitable for three-dimensional digital image correlation (3D-DIC) in large field of view (FOV). The method combines image feature algorithms with DIC techniques to extract matching point pairs (MPPs) from the left and right images of binocular cameras. These MPPs are then homogenized within the FOV. Next, initial values of the extrinsic parameters are solved based on epipolar constraint theory. Finally, the calibration parameters are nonlinearly optimized using the bundle adjustment method. To achieve stable and reliable numerical optimization in large FOV applications, the calibration control points (CCPs)’ spatial coordinates are represented using inverse depth parameterization. In scenarios where there might be a lack of sufficient CCPs, speckle patterns are artificially introduced to supplement the scene features. However, there is a lack of reliable experimental basis on how to add CCPs within the FOV. Therefore, through simulations, the factors affecting calibration accuracy are analyzed to guide the actual calibration process. The proposed method’s reliability and accuracy in large FOV 3D measurement are verified through experiments.

A Light Multi-View Stereo Method with Patch-Uncertainty Awareness.

Multi-view stereo methods utilize image sequences from different views to generate a 3D point cloud model of the scene. However, existing approaches often overlook coarse-stage features, impacting the final reconstruction accuracy. Moreover, using a fixed range for all the pixels during inverse depth sampling can adversely affect depth estimation. To address these challenges, we present a novel learning-based multi-view stereo method incorporating attention mechanisms and an adaptive depth sampling strategy. Firstly, we propose a lightweight, coarse-feature-enhanced feature pyramid network in the feature extraction stage, augmented by a coarse-feature-enhanced module. This module integrates features with channel and spatial attention, enriching the contextual features that are crucial for the initial depth estimation. Secondly, we introduce a novel patch-uncertainty-based depth sampling strategy for depth refinement, dynamically configuring depth sampling ranges within the GRU-based optimization process. Furthermore, we incorporate an edge detection operator to extract edge features from the reference image's feature map. These edge features are additionally integrated into the iterative cost volume construction, enhancing the reconstruction accuracy. Lastly, our method is rigorously evaluated on the DTU and Tanks and Temples benchmark datasets, revealing its low GPU memory consumption and competitive reconstruction quality compared to other learning-based MVS methods.

Analysis, synthesis, and rearrangement of 3D images by using multiview wavelets.

Multiview wavelets were designed to process images in autostereoscopic multiview three-dimensional (3D) displays. The structure of the proposed wavelets directly corresponds to the structure of the multiview image; the wavelets are made up of rectangular pulses. We demonstrated the analysis, synthesis, and rearrangement of 3D multiview images with the continuous wavelet transform, direct and inverse. By modifying wavelet coefficients, we made transformed (rearranged) 3D images (the rotation, the dimension of the parallax, the inverse depth, and the threshold). The visual 3D images confirm the correctness of processing and rearrangements. The simple structure of the proposed wavelets facilitates their practical application. The proposed technique can be applied to various (multiview/integral/light-field/plenoptic) 3D images.

Real-Time Pedestrian Detection with YOLOv7 and Intel MiDaS: A Qualitative Study

Abstract: In the realm of Advanced Driving Assistance Systems (ADAS), the accurate assessment of pedestrian proximity is of paramount importance. This paper introduces a qualitative methodology that integrates YOLOv7-pose for object detection and pose estimation with the MiDaS (Monocular Depth Estimation in Real-Time with Deep Learning) model for monocular depth estimation. The main objective is to qualitatively assess pedestrian proximity to the camera within the ADAS framework. This procedure involves classifying pedestrians as "near" or "far" based on an inverse depth threshold that has been predetermined. In addition, the paper performs a qualitative comparative analysis of the results produced by the MiDaS Small, Hybrid, and Large variants to learn more about the performance of depth estimation in these contexts, particularly in relation to the presence of pedestrians. The evaluation emphasises this approach's qualitative potential for achieving situationally appropriate and context-aware pedestrian proximity assessment. The safety and adaptability of ADAS systems can be improved with the help of such insights, which have numerous applications in robotics, surveillance, and autonomous vehicles.

Improving the Computational Efficiency of ROVIO

ROVIO is one of the state-of-the-art monocular visual inertial odometry algorithms. It uses an Iterative Extended Kalman Filter (IEKF) to align visual features and update the vehicle state simultaneously by including the feature locations in the state vector of the IEKF. This algorithm is single-core intensive, which allows the other cores to be used for other algorithms, such as object detection and path optimization. However, the computational cost of the algorithm grows rapidly with the maximum number of features to track. Each feature adds three new states (a 2D bearing vector and inverse depth), leading to bigger matrix multiplications that are computationally expensive. The main computational load of ROVIO is the iterative update step of the IEKF. In this work, we reduce the average computational cost of ROVIO by [Formula: see text] on an NVIDIA Jetson TX2, without affecting the accuracy of the algorithm. This computational gain is mainly achieved by exploiting the sparse matrices in ROVIO. Furthermore, we reduce the computational peaks by pre-selecting new features based on their already calculated FAST score. The combination of both modifications allows us to run ROVIO on the computationally restricted Raspberry Pi Zero 2[Formula: see text]W.

Sensor-Aided Calibration of Relative Extrinsic Parameters for Outdoor Stereo Vision Systems

Calibration of the stereo vision systems is a crucial step for precise 3D measurements. Restricted by the outdoors’ large field of view (FOV), the conventional method based on precise calibration boards is not suitable since the calibration process is time consuming and the calibration accuracy is not guaranteed. In this paper, we propose a calibration method for estimating the extrinsic parameters of the stereo vision system aided by an inclinometer and a range sensor. Through the parameters given by the sensors, the initial rotation angle of the extrinsic parameters and the translation vector are pre-established by solving a set of linear equations. The metric scale of the translation vector is determined by the baseline length provided by the range sensor or GNSS signals. Finally, the optimal extrinsic parameters of the stereo vision systems are obtained by nonlinear optimization of inverse depth parameterization. The most significant advantage of this method is that it enhances the capability of the stereo vision measurement in the outdoor environment, and can achieve fast and accurate calibration results. Both simulation and outdoor experiments have verified the feasibility and correctness of this method, and the relative error in the outdoor large FOV was less than 0.3%. It shows that this calibration method is a feasible solution for outdoor measurements with a large FOV and long working distance.

Integrating drilling parameters and machine learning tools to improve real-time porosity prediction of multi-zone reservoirs. Case study: Rhourd Chegga oilfield, Algeria

Porosity is a key variable for hydrocarbon reservoirs evaluation. It can be directly determined in laboratory tests using core samples or calculated indirectly from well logs. However, these methods are expensive and time consuming affecting the cost of supply of oil produced. Monitoring drilling variables in real time as a borehole is drilled can provide a cheap, initial insight to various rock formation properties. Investigating the relationships between rock porosity and drilling variables is therefore important. This study applies machines learning (ML) relate rock porosity to a range of measured drilling variables in several types of reservoir. Four ML models are evaluated: K-nearest neighbor (KNN), random forest (RF), support vector regression (SVR), and eXtreme Gradient Boosting (XGBoost). A measured drilling and porosity dataset from the Rhourd Chegga oilfield (SE Algeria) is compiled from its multi-zone reservoir system. The drilling variables from nine vertical wellbores considered are: weight on bit, torque, standpipe pressure, drill string rotation speed, rate of penetration and its inverse, pump rate and bit depth. The recorded data is derived from three distinct reservoir zones: Trias (T1), Rhourd Chegga sandstone (RDC), and the Quartzite Hamra tight reservoir (QH). Porosity predictions were derived in two ways: (1) for each reservoir separately; (2) for the three reservoirs combined. Prediction error analysis was conducted in terms of the coefficient of determination (R2), root mean squared error (RMSE) and mean absolute error (MAE). The XGBoost model yielded the best porosity predictions for the individual and combined reservoirs. Prediction errors generated for reservoir T1 were (R2 ≈ 0.865, RMSE≈0.866, MAE≈0.692); for reservoir RDC were R2 ≈ 0.935, RMSE≈0.848, and MAE≈0.772; and, for reservoir QH were R2 ≈ 0.743, RMSE≈0.794, and MAE≈0.573. For the combined (multi-zone) reservoir (T1+RDC + QH), porosity predictions were less accurate, and the XGBoost model generated prediction errors of R2 ≈ 0.76, RMSE≈1.259, and MAE≈1.027.

Method for 2D-3D Registration under Inverse Depth and Structural Semantic Constraints for Digital Twin City

A digital twin city maps a virtual three-dimensional (3D) city model to the geographic information system, constructs a virtual world, and integrates real sensor data to achieve the purpose of virtual–real fusion. Focusing on the accuracy problem of vision sensor registration in the virtual digital twin city scene, this study proposes a 2D-3D registration method under inverse depth and structural semantic constraints. First, perspective and inverse depth images of the virtual scene were obtained by using perspective view and inverse-depth nascent technology, and then the structural semantic features were extracted by the two-line minimal solution set method. A simultaneous matching and pose estimation method under inverse depth and structural semantic constraints was proposed to achieve the 2D-3D registration of real images and virtual scenes. The experimental results show that the proposed method can effectively optimize the initial vision sensor pose and achieve high-precision registration in the digital twin scene, and the Z-coordinate error is reduced by 45%. An application experiment of monocular image multi-object spatial positioning was designed, which proved the practicability of this method, and the influence of model data error on registration accuracy was analyzed.

Considerations for Upright Particle Therapy Patient Positioning and Associated Image Guidance.

Particle therapy is a rapidly growing field in cancer therapy. Worldwide, over 100 centers are in operation, and more are currently in construction phase. The interest in particle therapy is founded in the superior target dose conformity and healthy tissue sparing achievable through the particles’ inverse depth dose profile. This physical advantage is, however, opposed by increased complexity and cost of particle therapy facilities. Particle therapy, especially with heavier ions, requires large and costly equipment to accelerate the particles to the desired treatment energy and steer the beam to the patient. A significant portion of the cost for a treatment facility is attributed to the gantry, used to enable different beam angles around the patient for optimal healthy tissue sparing. Instead of a gantry, a rotating chair positioning system paired with a fixed horizontal beam line presents a suitable cost-efficient alternative. Chair systems have been used already at the advent of particle therapy, but were soon dismissed due to increased setup uncertainty associated with the upright position stemming from the lack of dedicated image guidance systems. Recently, treatment chairs gained renewed interest due to the improvement in beam delivery, commercial availability of vertical patient CT imaging and improved image guidance systems to mitigate the problem of anatomical motion in seated treatments. In this review, economical and clinical reasons for an upright patient positioning system are discussed. Existing designs targeted for particle therapy are reviewed, and conclusions are drawn on the design and construction of chair systems and associated image guidance. Finally, the different aspects from literature are channeled into recommendations for potential upright treatment layouts, both for retrofitting and new facilities.

Radio Emission of Pulsars. II. Coherence Catalyzed by Cerenkov-unstable Shear Alfvén Waves

This paper explores small-scale departures from force-free electrodynamics around a rotating neutron star, extending our treatment of resistive instability in a quantizing magnetic field. A secondary, Cerenkov instability is identified: relativistic particles flowing through thin current sheets excite propagating charge perturbations that are localized near the sheets. Growth is rapid at wavenumbers below the inverse ambient skin depth k p,ex. Small-scale Alfvénic wavepackets are promising sources of coherent curvature radiation. When the group Lorentz factor , where R c is the magnetic curvature radius, a fraction ∼10−3–10−2 of the particle kinetic energy is radiated into the extraordinary mode at a peak frequency ∼10−2 ck p,ex. Consistency with observations requires a high pair multiplicity (∼103–5) in the pulsar magnetosphere. Neither the primary, slow resistive instability nor the secondary, Alfvénic instability depend directly on the presence of magnetospheric gaps, and may activate where the mean current is fully supplied by outward drift of the corotation charge. The resistive mode is overstable and grows at a rate comparable to the stellar spin frequency; the model directly accommodates strong pulse-to-pulse radio flux variations and coordinated subpulse drift. Alfvén mode growth can track the local plasma conditions, allowing for lower-frequency emission from the outer magnetosphere. Beamed radio emission from charged packets with γ gr ∼ 50–100 also varies on submillisecond timescales. The modes identified here will be excited inside the magnetosphere of a magnetar, and may mediate Taylor relaxation of the magnetic twist.

Refinement of Inverse Depth Plane in Textureless and Occluded Regions in a Multiview Stereo Matching Scheme

In the multiview stereo (MVS) vision, it is difficult to estimate accurate depth in the textureless and occluded regions. To solve this problem, several MVS investigations employ the matching cost volume (MCV) approach to refine the cost in the textureless and occluded regions. Usually, the matching costs in the large textureless image regions are not reliable. In addition, if an occluded region is also textureless, the matching cost contains a significant error. The goal of the proposed MVS method is to reconstruct accurate depth maps in both large-textureless and occluded-textureless regions by iteratively updating the erroneous disparity. The erroneous disparity in the textureless region is updated by the 3D disparity plane of the region in the inverse-depth space. Then, the surface consensus is computed and used to run the two processes, the surface consensus refinement and the matching cost update. By the iterative update of the 3D inverse depth plane, surface consensus, and matching cost, the performance of the depth reconstruction in the large-textureless and occluded-textureless regions is greatly improved. The performance of the proposed method is analyzed using the Middlebury multiview stereo dataset. The depth reconstruction performance is also compared with several stereo vision methods.

A Method of Inversing Dynamic Aerosol Extinction-to-Backscattering Ratio Based on Lidar Echo Signal and Ground Aerosol Extinction Coefficient or Aerosol Optical Depth

When using the Fernald method to solve the lidar equation, the selection of aerosol extinction-to-backscattering ratio (lidar ratio) is an important error source, which becomes a major factor limiting the accuracy of aerosol extinction profile. Therefore, we proposed a method for dynamic lidar ration inversion, named iteration inverse generation method (IIGM). This method is based on the Fernald method, takes the ground extinction coefficient as the loop iteration constraints, and is combined with the lidar signal to inverse the local dynamic lidar ratio. The local lidar ratio obtained by IIGM is in the range of 10–50 sr. The average relative error between the ground extinction coefficient inversed from dynamic lidar ratio and the measured ground extinction coefficient is 0.0041%. The relative error between the inverse aerosol optical depth (AOD) and the measured AOD is 20.38%. In addition to using the ground extinction coefficient as the iteration constraints of IIGM, AOD is introduced as another optional iterative constraint. The IIGM can achieve accurate inversion of lidar signals in different situations.

Ego-Motion Estimation With Stereo Cameras Using Efficient 3D–2D Edge Correspondences

This article presents a new edge-based ego-motion estimation method for stereo cameras using 3D–2D edge alignment. To filter out prominent edgelets from Canny edge extractor, a region growing algorithm is developed to remove cluttered edges out of raw edgelets. Edge matching between stereo image pairs is guided by sparse feature correspondence using optical flow. The disparity is then refined using an inverse compositional algorithm, which is far more computationally efficient than searching along the epipolar line. The relative camera motion is solved by 3D–2D edge alignment based on an iterative closest point (ICP) algorithm, minimizing the point-to-tangent distance. A gradient-orientation-based similarity measurement is proposed, and the optimal correspondence is searched between the 3-D edge point projected onto the current image and the extracted edge pixels in the current frame with 1-nearest neighbor. In addition, the inverse depth of edge points in the reference frame is optimized. Experiments show that the proposed method can provide centimeter-level relative pose error (RPE), which has a better performance than the point-based approach efficient perspective-n-point (EPnP) and the edge-based approach direct edge-alignment (DE-A).

Dynamic SLAM of binocular catadioptric panoramic camera based on inverse depth filter

Dynamic SLAM of binocular catadioptric panoramic camera based on inverse depth filter

Three-Filters-to-Normal: An Accurate and Ultrafast Surface Normal Estimator

This paper proposes three-filters-to-normal (3F2N), an accurate and ultrafast surface normal estimator (SNE), which is designed for structured range sensor data, e.g., depth/disparity images. 3F2N SNE computes surface normals by simply performing three filtering operations (two image gradient filters in horizontal and vertical directions, respectively, and a mean/median filter) on an inverse depth image or a disparity image. Despite the simplicity of 3F2N SNE, no similar method already exists in the literature. To evaluate the performance of our proposed SNE, we created three large-scale synthetic datasets (easy, medium and hard) using 24 3D mesh models, each of which is used to generate 1800--2500 pairs of depth images (resolution: 480X640 pixels) and the corresponding ground-truth surface normal maps from different views. 3F2N SNE demonstrates the state-of-the-art performance, outperforming all other existing geometry-based SNEs, where the average angular errors with respect to the easy, medium and hard datasets are 1.66 degrees, 5.69 degrees and 15.31 degrees, respectively. Furthermore, our C++ and CUDA implementations achieve a processing speed of over 260 Hz and 21 kHz, respectively. Our datasets and source code are publicly available at sites.google.com/view/3f2n.

Spin waves and high-frequency response in layered superconductors with helical magnetic structure

We evaluate the spin-wave spectrum and dynamic susceptibility in a layered superconductors with helical interlayer magnetic structure. We especially focus on the structure in which the moments rotate 90$^{\circ}$ from layer to layer realized in the iron pnictide RbEuFe$_{4}$As$_{4}$. The spin-wave spectrum in superconductors is strongly renormalized due to the long-range electromagnetic interactions between the oscillating magnetic moments. This leads to strong enhancement of the frequency of the mode coupled with uniform field and this enhancement exists only within a narrow range of the c-axis wave vectors of the order of the inverse London penetration depth. The key feature of materials like RbEuFe$_{4}$As$_{4}$ is that this uniform mode corresponds to the maximum frequency of the spin-wave spectrum with respect to c-axis wave vector. As a consequence, the high-frequency surface resistance acquires a very distinct asymmetric feature spreading between the bare and renormalized frequencies. We also consider excitation of spin waves with Josephson effect in a tunneling contact between helical-magnetic and conventional superconductors and study the interplay between the spin-wave features and geometrical cavity resonances in the current-voltage characteristics.

DeepFusion: A simple way to improve traditional multi-view stereo methods using deep learning

In this paper, we propose a simple deep learning based method, “DeepFusion”, to improve the results from traditional multi-view stereo (MVS) methods. In our method, we first design a deep convolutional neural network to predict the confidences of the depths produced by the traditional MVS method, which takes the colors, depths, normals and costs as inputs. In particular, we convert the depths into the normalized inverse depths to enhance the accuracy and generality of the network. In addition, we use the plane sweeps in the disparity space to get the top-k matching costs in order to determine the distinctiveness among all costs. Then we propose a novel approach to accurately fuse all depth maps into the final point cloud, which balances the geometric consistency and the predicted confidences. Experimental results on several well-known datasets demonstrate the effectiveness and the generalization ability of the proposed method.