Inverse Depth Research Articles

Assessing the Nutrient Dynamics in a Himalayan Warm Monomictic Lake

Thermal stratification is considered to be the most important limnological feature of deep lake ecosystems affecting a water column’s chemical characteristics thereby directly connected with lake management. The lake is also attracting the attention of management authorities, being a growing tourist destination with sustainable environmental management plan. Lake Manasbal is only true warm monomictic urban lake in Kashmir and remains thermally stratified for 8–9 months, with separate layers of epilimnion, metalimnion and hypolimnion.We noticed that in the lower part of the hypolimnion, the process of stratification is characterized by an oxygen deficit (O2 < 1 mg/L). The rate of oxygen depletion reported in the lake’s bottom layers is indicative of the water body’s more eutrophic nature. A clinograde type of pH curve with a wide range of fluctuation from top to bottom layers of water was observed. Thermal stratification of the lake allows ammonium and phosphates to pool in the hypolimnion for much of the stratification period, resulting in concentrations higher than in the epilimnion. Consequently, the lake was characterized by a nutrient-enriched hypolimnion and an epilimnion deprived of nutrients. With the advent of the lake stratification, the thermocline was formed on 15th March between 2 and 3 m and on 7th April between 4 and 6 m, which gradually moved down to 10 m on 4th July and remained at that depth until circulation. The concentration of phosphorus and nitrogen in Lake Manasbal showed an increase towards the bottom, thus revealing an inverse clinograde depth distribution. Due to stratification phenomena in this lake, the internal release and subsequent distribution of nutrients throughout water column especially phosphorus from sediment water interface after mixing is a cause of concern for management authorities as it has the potential to diminish the ecological and aesthetics of the lake.Therefore, we argue that this study will provide some valuable insights on how stratification is affecting the nutrient dynamics and will offer inputs for better management accordingly both from ecological and economic perspectives.

A Plane Extraction Approach in Inverse Depth Images Based on Region-Growing.

Planar surfaces are prevalent components of man-made indoor scenes, and plane extraction plays a vital role in practical applications of computer vision and robotics, such as scene understanding, and mobile manipulation. Nowadays, most plane extraction methods are based on reconstruction of the scene. In this paper, plane representation is formulated in inverse-depth images. Based on this representation, we explored the potential to extract planes in images directly. A fast plane extraction approach, which employs the region growing algorithm in inverse-depth images, is presented. This approach consists of two main components: seeding, and region growing. In the seeding component, seeds are carefully selected locally in grid cells to improve exploration efficiency. After seeding, each seed begins to grow into a continuous plane in succession. Both greedy policy and a normal coherence check are employed to find boundaries accurately. During growth, neighbor coplanar planes are checked and merged to overcome the over-segmentation problem. Through experiments on public datasets and generated saw-tooth images, the proposed approach achieves 80.2% CDR (Correct Detection Rate) on the ABW SegComp Dataset, which has proven that it has comparable performance with the state-of-the-art. The proposed approach runs at 5 Hz on typical 680 × 480 images, which has shown its potential in real-time practical applications in computer vision and robotics with further improvement.

ScPnP: A non-iterative scale compensation solution for PnP problems

This paper presents an accurate non-iterative method for the Perspective-n-Point problem(PnP). Our main idea is to mitigate scale bias by multiplying an independent inverse average depth variable onto the object space error. The introduced variable is of order 2 in the objective function and the optimality conditions constitute a polynomial system with three third-order and one first-order unknowns. Subsequently, we employ the Dixon resultant method to compute explicit expressions of the action matrix, the eigenvalue decomposition of which determines all the roots of the problem. We further extend this scale compensation technology to sphere cameras and contribute a uniform solver to PnP problems for both camera types. Experimental results confirm that our method is reliable and more accurate than state-of-the-art PnP algorithms.

Full- and Reduced-Order Observers for Image-Based Depth Estimation Using Concurrent Learning

In this brief, concurrent learning (CL)-based full- and reduced-order observers for a perspective dynamical system (PDS) are developed. The PDS is a widely used model for estimating the depth of a feature point from a sequence of camera images. Building on the current progress of CL for parameter estimation in adaptive control, state observers are developed for the PDS model where the inverse depth appears as a time-varying parameter in the dynamics. The data recorded over a sliding time window in the near past are used in the CL term to design the full- and reduced-order state observers. A Lyapunov-based stability analysis is carried out to prove the uniformly ultimately bounded (UUB) stability of the developed observers. Simulation results are presented to validate the accuracy and convergence of the developed observers in terms of convergence time, root mean square error (RMSE), and mean absolute percentage error (MAPE) metrics. Real-world depth estimation experiments are performed to demonstrate the performance of the observers using the aforementioned metrics on a 7-DoF manipulator with an eye-in-hand configuration.

3D Semantic VSLAM of Indoor Environment Based on Mask Scoring RCNN

In view of existing Visual SLAM (VSLAM) algorithms when constructing semantic map of indoor environment, there are problems with low accuracy and low label classification accuracy when feature points are sparse. This paper proposed a 3D semantic VSLAM algorithm called BMASK-RCNN based on Mask Scoring RCNN. Firstly, feature points of images are extracted by Binary Robust Invariant Scalable Keypoints (BRISK) algorithm. Secondly, map points of reference key frame are projected to current frame for feature matching and pose estimation, and an inverse depth filter is used to estimate scene depth of created key frame to obtain camera pose changes. In order to achieve object detection and semantic segmentation for both static objects and dynamic objects in indoor environments and then construct dense 3D semantic map with VSLAM algorithm, a Mask Scoring RCNN is used to adjust its structure partially, where a TUM RGB-D SLAM dataset for transfer learning is employed. Semantic information of independent targets in scenes provides semantic information including categories, which not only provides high accuracy of localization but also realizes the probability update of semantic estimation by marking movable objects, thereby reducing the impact of moving objects on real-time mapping. Through simulation and actual experimental comparison with other three algorithms, results show the proposed algorithm has better robustness, and semantic information used in 3D semantic mapping can be accurately obtained.

Lightweight hybrid visual-inertial odometry with closed-form zero velocity update

Visual-Inertial Odometry (VIO) fuses measurements from camera and Inertial Measurement Unit (IMU) to achieve accumulative performance that is better than using individual sensors. Hybrid VIO is an extended Kalman filter-based solution which augments features with long tracking length into the state vector of Multi-State Constraint Kalman Filter (MSCKF). In this paper, a novel hybrid VIO is proposed, which focuses on utilizing low-cost sensors while also considering both the computational efficiency and positioning precision. The proposed algorithm introduces several novel contributions. Firstly, by deducing an analytical error transition equation, one-dimensional inverse depth parametrization is utilized to parametrize the augmented feature state. This modification is shown to significantly improve the computational efficiency and numerical robustness, as a result achieving higher precision. Secondly, for better handling of the static scene, a novel closed-form Zero velocity UPdaTe (ZUPT) method is proposed. ZUPT is modeled as a measurement update for the filter rather than forbidding propagation roughly, which has the advantage of correcting the overall state through correlation in the filter covariance matrix. Furthermore, online spatial and temporal calibration is also incorporated. Experiments are conducted on both public dataset and real data. The results demonstrate the effectiveness of the proposed solution by showing that its performance is better than the baseline and the state-of-the-art algorithms in terms of both efficiency and precision. A related software is open-sourced to benefit the community.①https://github.com/PetWorm/LARVIO①

Learning Inverse Depth Regression for Multi-View Stereo with Correlation Cost Volume

Deep learning has shown to be effective for depth inference in multi-view stereo (MVS). However, the scalability and accuracy still remain an open problem in this domain. This can be attributed to the memory-consuming cost volume representation and inappropriate depth inference. Inspired by the group-wise correlation in stereo matching, we propose an average group-wise correlation similarity measure to construct a lightweight cost volume. This can not only reduce the memory consumption but also reduce the computational burden in the cost volume filtering. Based on our effective cost volume representation, we propose a cascade 3D U-Net module to regularize the cost volume to further boost the performance. Unlike the previous methods that treat multi-view depth inference as a depth regression problem or an inverse depth classification problem, we recast multi-view depth inference as an inverse depth regression task. This allows our network to achieve sub-pixel estimation and be applicable to large-scale scenes. Through extensive experiments on DTU dataset and Tanks and Temples dataset, we show that our proposed network with Correlation cost volume and Inverse DEpth Regression (CIDER1), achieves state-of-the-art results, demonstrating its superior performance on scalability and accuracy.

Peat decomposition proxies of Alpine bogs along a degradation gradient

The carbon sink function of peatlands is the result of a relatively small imbalance between biomass production and decay and proposed to be sensitive to changes in environmental conditions. Thus, peat decomposition plays a decisive role in determining the future carbon balances of peatlands under changing climatic conditions and increasing human pressure. To evaluate the effects of peatland degradation on the degree of decomposition in light of deviating site histories and to reveal interrelations between different indicators, we analysed peat depth profiles of physicochemical peat decomposition proxies across nine study sites in two valley bogs in the Eastern Alps, Austria. For this purpose, we divided 1 m peat cores sampled at the nine sites into 4 cm sections and analysed these peat samples for bulk density, loss on ignition, pH value, carbon and nitrogen content and ratio, stable carbon and nitrogen isotope signatures, and five humification indices based on Fourier transform infrared spectroscopy. Besides, we regularly measured water levels and incubated litter bags over two years. Peat characteristics were found to be relatively homogenous across the treeless and only slightly degraded sites. At these sites, depth profiles followed similar patterns with strongly increasing decomposition in the uppermost layers and the highest degree of peat decomposition at 40–56 cm depths for most proxies. Thus, our findings suggest that even apparently only slightly degraded sites were distinctly affected by past drainages. In contrast to the open peat bog sites, the degree in peat decomposition was distinctly higher at tree-encroached sites and at long-term drained former peat-cutting sites. Both, the distinct difference in the degree of decomposition between the two peatlands and the inverse decomposition depth trends, i.e., more decomposed peat overlying less decomposed layers, highlight the importance of secondary decomposition on the depth profiles and the strong effects of site histories on the current degree of peat decomposition. The results of two-year litter bag incubations of four vascular plant species revealed distinct differences in species-specific litter decomposition rates and considerable differences for same species between the two investigated bogs. The strong interrelations between different humification indices underline the simultaneous effects of proceeding decay on the different chemical properties of the peat. Besides, humification indices were strongly related with other physical and chemical properties of peats and highest loadings on the principal component 1 that accounted for 41% of the total variance of the dataset were found for four different humification indices, bulk density and nitrogen contents. We conclude that the selective preservation of aliphatic, acid and aromatic groups during peat decomposition are well suited to elucidate small-scale differences in the degree of decomposition and to strongly interrelate with established indicators as bulk densities.

Autonomous State Estimation and Mapping in Unknown Environments With Onboard Stereo Camera for Micro Aerial Vehicles

Industrial micro aerial vehicles (MAVs) with robotic manipulators have numerous applications in search and rescue tasks that reduce risks to human beings. However, such tasks distinctly require MAVs to have the capability of real-time autonomous navigation only with onboard sensors, especially in GPS-denied applications. This article introduces a new approach to onboard vision-based autonomous state estimation and mapping for MAVs’ navigation in unknown environments. The algorithms run on board and do not need an external positioning system to assist autonomous navigation. The state estimator is developed to provide MAV's current pose on the basis of the extended Kalman filter by using image patch features. Inverse depth convergence monitoring and local bundle adjustment are utilized to improve the accuracy. The mapping algorithm for navigation is developed according to a real-time stereo matching method for three-dimensional perception. Finally, we have performed several experiments to demonstrate the effectiveness of the proposed approach.

Joint optimization based on direct sparse stereo visual-inertial odometry

This paper proposes a novel fusion of an inertial measurement unit (IMU) and stereo camera method based on direct sparse odometry (DSO) and stereo DSO. It jointly optimizes all model parameters within a sliding window, including the inverse depth of all selected pixels and the internal or external camera parameters of all keyframes. The vision part uses a photometric error function that optimizes 3D geometry and camera pose in a combined energy functional. The proposed algorithm uses image blocks to extract neighboring image features and directly forms measurement residuals in the image intensity space. A fixed-baseline stereo camera solves scale drift. IMU information is accumulated between several frames using manifold pre-integration and is inserted into the optimization as additional constraints between keyframes. The scale and gravity inserted are incorporated into the stereo visual inertial odometry model and are optimized together with other variables such as poses. The experimental results show that the tracking accuracy and robustness of the proposed method are superior to those of the state-of-the-art fused IMU method. In addition, compared with previous semi-dense direct methods, the proposed method displays a higher reconstruction density and scene recovery.

Multitask Deep Neural Networks for Tele-Wide Stereo Matching

In this paper, we introduce the problem of estimating the real world depth of elements in a scene captured by two cameras with different field of views, where the first field of view (FOV) is a Wide FOV (WFOV) captured by a wide angle lens, and the second FOV is contained in the first FOV and is captured by a tele zoom lens. We refer to the problem of estimating the inverse depth for the union of FOVs, while leveraging the stereo information in the overlapping FOV, as Tele-Wide Stereo Matching (TW-SM). We propose different deep learning solutions to the TW-SM problem. Since the disparity is proportional to the inverse depth, we train stereo matching disparity estimation (SMDE) networks to estimate the disparity for the union WFOV. We further propose an end-to-end deep multitask tele-wide stereo matching neural network (MT-TW-SMNet), which simultaneously learns the SMDE task for the overlapped Tele FOV and the single image inverse depth estimation (SIDE) task for the WFOV. Moreover, we design multiple methods for the fusion of the SMDE and SIDE networks. We evaluate the performance of TW-SM on the popular KITTI and SceneFlow stereo datasets, and demonstrate its practicality by synthesizing the Bokeh effect on the WFOV from a tele-wide stereo image pair.

Integrated Autonomous Relative Navigation Method Based on Vision and IMU Data Fusion

An integrated autonomous relative navigation method based on vision and IMU data fusion was proposed in this paper, which can improve the position accuracy effectively and has strong adaptability to environmental changes. Firstly, IMU pre-integration formula based on Runge Kutta method was derived, which can improve the pre-integration position accuracy and reduce the accumulated error effectively. Secondly, an inverse depth estimation method based on the mixed probability model was proposed during the system initialization process, which can improve the accuracy of camera depth estimation and provide better initial conditions for back-end optimization. Thirdly, a sliding window filtering method based on the probability graph was proposed, which can avoid repeated calculations and improve the sliding window filtering efficiency. Forthly, combined with the advantages of the direct method and the feature point method, a mixed re-projection optimization method was proposed, which can expand the application scope of the method and improve the optimization accuracy effectively. Finally, in the closed-loop optimization, a closed-loop optimization method based on similar transformation is proposed to eliminate the accumulated error. In order to verify the environmental adaptability of the method and the impact of closed-loop detection on the relative navigation system, indoor and outdoor experiments were carried out with a hand-held camera and an IMU. EuRoC dataset was used in the experiments and the proposed method was compared with some classical methods. The experimental results showed that this method has high accuracy and robustness.

SDF-SLAM: Semantic Depth Filter SLAM for Dynamic Environments

Simultaneous Localization and Mapping (SLAM) has been widely applied in computer vision and robotics. For the dynamic environments which are very common in the real word, traditional visual SLAM system faces significant drop in localization and mapping accuracy due to the static world assumption. Recently, the semantic visual SLAM systems towards dynamic scenes have gradually attracted more and more attentions, which use the semantic information of images to help remove dynamic feature points. Existing semantic visual SLAM systems commonly detect the dynamic feature points by the semantic prior, geometry constraint or the combine of them, then map points corresponding to dynamic feature points are removed. In the visual SLAM framework, pose calculation is essentially around the 3D map points, so the essence of improving the accuracy of visual SLAM system is to build a more accurate and reliable map. These existing semantic visual SLAM systems are actually adopting an indirect way to acquire reliable map points, and several drawbacks exist. In this paper, we present SDF-SLAM: Semantic Depth Filter SLAM, a visual semantic SLAM system towards dynamic environments, which utilizes the technology of depth filter to directly judge whether a 3D map point is dynamic or not. First, the semantic information is integrated into the original pure geometry SLAM system by the semantic optical flow method to perform reliable map initialization. Second, design the semantic depth filter that satisfies the Gaussian Uniform mixture distribution to describe the inverse depth of each map point. Third, updating the inverse depth of 3D map point in a Bayesian estimation framework, and dividing the 3D map point into active one or inactive one. Last, only the active map points are utilized to achieve robust camera pose tracking. Experiments on TUM dataset demonstrate that our approach outperforms original ORB-SLAM2 and other state-of-the-art semantic SLAM systems.

基于逆深度自适应加权的多视图三角化方法

基于逆深度自适应加权的多视图三角化方法

Relativistic Approach for Training Self-Supervised Adversarial Depth Prediction Model Using Symmetric Consistency

This article proposes a novel approach for predicting depth from a single image in a self-supervised manner, specifically using generative adversarial networks (GANs) with enhancements to improve training performance. We train our model to generate disparity maps or inverse depth maps using left/right stereo image supervision rather than direct supervision from depth ground truth. We avoid the problem of training instability in standard GANs by employing a relativistic discriminator that evaluates how realistic an input image is. Since the reconstructed image quality determines the disparity map's accuracy, we adopt the GMSD, one of the standout IQA algorithms, as a part of the image reconstruction loss function. A VGG-19 pre-trained model-based perceptual loss is used to optimize our model in a feature space instead of pixel space and to enhance reconstructed images' perceptual quality with the learning-based perspective. Also, we add another symmetric generator that takes the right images as input, which helps to maintain the consistency of left/right views and enrich our training data with reasonable overhead on the training time. Our network is trained to predict depths of 640 × 192 images using the KITTI stereo dataset, which shows an outstanding performance compared to state-of-the-art works in various evaluation metrics. We verify our method's modules by performing an ablation study and evaluate the generalization performance on CityScapes and Make3D datasets.

Incremental Pose Map Optimization for Monocular Vision SLAM Based on Similarity Transformation.

The novel contribution of this paper is to propose an incremental pose map optimization for monocular vision simultaneous localization and mapping (SLAM) based on similarity transformation, which can effectively solve the scale drift problem of SLAM for monocular vision and eliminate the cumulative error by global optimization. With the method of mixed inverse depth estimation based on a probability graph, the problem of the uncertainty of depth estimation is effectively solved and the robustness of depth estimation is improved. Firstly, this paper proposes a method combining the sparse direct method based on histogram equalization and the feature point method for front-end processing, and the mixed inverse depth estimation method based on a probability graph is used to estimate the depth information. Then, a bag-of-words model based on the mean initialization K-means is proposed for closed-loop feature detection. Finally, the incremental pose map optimization method based on similarity transformation is proposed to process the back end to optimize the pose and depth information of the camera. When the closed loop is detected, global optimization is carried out to effectively eliminate the cumulative error of the system. In this paper, indoor and outdoor environmental experiments are carried out using open data sets, such as TUM and KITTI, which fully proves the effectiveness of this method. Closed-loop detection experiments using hand-held cameras verify the importance of closed-loop detection. This method can effectively solve the scale drift problem of monocular vision SLAM and has strong robustness.

Histograms of the Normalized Inverse Depth and Line Scanning for Urban Road Detection

In this paper, we propose to fuse the geometric information of a 3-D LiDAR and a monocular camera to detect the urban road region ahead of an autonomous vehicle. Our method takes advantage of both the high definition of 3-D LiDAR data and the continuity of road in image representation. First, we obtain an efficient representation of LiDAR data and an organized 2-D inverse depth map, by projecting the 3-D LiDAR points onto the camera's image plane. Through the new representation, we can acquire the intermediate representations of road scenes by extracting the vertical and horizontal histograms of the normalized inverse depth. The approximate road regions can be quickly estimated with both histogram-based schemes. To accurately find the road area, we propose a row and column scanning strategy in the approximate road region to refine the detected road area. We have carried out experiments on the public KITTI-Road benchmark, and have achieved one of the best performances among the LiDAR-based road detection methods without learning procedure.

Online Spatial and Temporal Calibration for Monocular Direct Visual-Inertial Odometry.

Owing to the nonlinearity in visual-inertial state estimation, sufficiently accurate initial states, especially the spatial and temporal parameters between IMU (Inertial Measurement Unit) and camera, should be provided to avoid divergence. Moreover, these parameters are required to be calibrated online since they are likely to vary once the mechanical configuration slightly changes. Recently, direct approaches have gained popularity for their better performance than feature-based approaches in little-texture or low-illumination environments, taking advantage of tracking pixels directly. Based on these considerations, we perform a direct version of monocular VIO (Visual-inertial Odometry), and propose a novel approach to initialize the spatial-temporal parameters and estimate them with all other variables of interest (IMU pose, point inverse depth, etc.). We highlight that our approach is able to perform robust and accurate initialization and online calibration for the spatial and temporal parameters without utilizing any prior information, and also achieves high-precision estimates even when large temporal offset occurs. The performance of the proposed approach was verified through the public UAV (Unmanned Aerial Vehicle) dataset.

A Method of Target Tracking Based on Monocular Vision for Mobile Robot in Unknown Environment

In order to solve the problem of target tracking based on monocular vision for mobile robot in unknown environment, a robot simultaneous localization, map building and target tracking method is proposed. This method can realize the simultaneous online estimation of robot, environment features and target states by using the bearings only observations from monocular sensors. In the process of estimation, the monocular vision based Simultaneous localization and mapping (MVSLAM) runs independent of the Object Tracking(OT), as the basis of the OT, MVSLAM provides the state information of the robot platform for the OT, and OT uses the bearing measurements of target and robot state to estimates the state of target. MVSLAM is based on inverse depth parameterization and running under framework of full probability Extended Kalman Filter (EKF), and another EKF is also used for target tracking independently. The simulation experiments show the performance of the design method, and analyze the problem of observability of Bearing-only tracking. To solve the observability problem, a robot motion control method is presented, under the control of this method, the robot will make satellite surround motion around the target. Finally, the feasibility and accuracy of the method are verified by simulation experiments.

Auto-calibration and real-time external parameter correction for stereo digital image correlation

Calibrating stereo digital image correlation (stereo-DIC) is crucial for 3D deformation measurements. Existing stereo-DIC calibration methods mainly rely on a well-made planar target with additional specific coding to perform pre-calibration. In this paper, a new method is proposed to calibrate a stereo-DIC system automatically with feature correspondences in an unconstrained scene. In contrast to traditional methods, the proposed method is scale-independent and does not require assumptions such as planarity. More importantly, it is capable of correcting disturbed external parameters in real time. With inverse depth parameterization, a stereo-DIC system can be effectively calibrated by a carefully designed bundle adjustment framework. Subsequently, a fast correction strategy based on massive-point subset tracking is derived to recalibrate the disturbed stereo-DIC system. The proposed method overcomes the predicament of reduced measurement accuracy due to the unavoidable disturbance of external parameters in dynamic deformation measurements. Experiments were conducted to verify the effectiveness of the proposed method.