Depth Estimation Research Articles

Snow depth sensitivity to mean temperature, precipitation, and elevation in the Austrian and Swiss Alps

Abstract. Snow depth plays an important role in the seasonal climatic and hydrological cycles of alpine regions. Previous studies have shown predominantly decreasing trends in average seasonal snow depth across the European Alps. Additionally, prior work has shown bivariate statistical relationships between average seasonal snow depth and mean air temperature or precipitation. Building upon existing research, our study uses observational records of in situ station data across Austria and Switzerland to better quantify the sensitivity of historical changes in seasonal snow depth through a multivariate framework that depends on elevation, mean temperature, and precipitation. These historical sensitivities, which are obtained over the 1901–1902 to 1970–1971 period, are then used to estimate snow depths over the more recent period of 1971–1972 to 2020–2021. We find that the year-to-year estimates of snow depths, which are derived from an empirical–statistical model (SnowSens), that rely solely on the historical sensitivities are nearly as skillful as the operational SNOWGRID-CL model used by the weather service at GeoSphere Austria. Furthermore, observed long-term changes over the last 50 years are in better agreement with SnowSens than with SNOWGRID-CL. These results indicate that historical sensitivities between snow depth, temperature, and precipitation are quite robust over decadal-length scales of time, and they can be used effectively to translate expected long-term changes in temperature and precipitation into changes in seasonal snow depth.

The Depth Estimation and Visualization of Dermatological Lesions: Development and Usability Study.

Thus far, considerable research has been focused on classifying a lesion as benign or malignant. However, there is a requirement for quick depth estimation of a lesion for the accurate clinical staging of the lesion. The lesion could be malignant and quickly grow beneath the skin. While biopsy slides provide clear information on lesion depth, it is an emerging domain to find quick and noninvasive methods to estimate depth, particularly based on 2D images. This study proposes a novel methodology for the depth estimation and visualization of skin lesions. Current diagnostic methods are approximate in determining how much a lesion may have proliferated within the skin. Using color gradients and depth maps, this method will give us a definite estimate and visualization procedure for lesions and other skin issues. We aim to generate 3D holograms of the lesion depth such that dermatologists can better diagnose melanoma. We started by performing classification using a convolutional neural network (CNN), followed by using explainable artificial intelligence to localize the image features responsible for the CNN output. We used the gradient class activation map approach to perform localization of the lesion from the rest of the image. We applied computer graphics for depth estimation and developing the 3D structure of the lesion. We used the depth from defocus method for depth estimation from single images and Gabor filters for volumetric representation of the depth map. Our novel method, called red spot analysis, measures the degree of infection based on how a conical hologram is constructed. We collaborated with a dermatologist to analyze the 3D hologram output and received feedback on how this method can be introduced to clinical implementation. The neural model plus the explainable artificial intelligence algorithm achieved an accuracy of 86% in classifying the lesions correctly as benign or malignant. For the entire pipeline, we mapped the benign and malignant cases to their conical representations. We received exceedingly positive feedback while pitching this idea at the King Edward Memorial Institute in India. Dermatologists considered this a potentially useful tool in the depth estimation of lesions. We received a number of ideas for evaluating the technique before it can be introduced to the clinical scene. When we map the CNN outputs (benign or malignant) to the corresponding hologram, we observe that a malignant lesion has a higher concentration of red spots (infection) in the upper and deeper portions of the skin, and that the malignant cases have deeper conical sections when compared with the benign cases. This proves that the qualitative results map with the initial classification performed by the neural model. The positive feedback provided by the dermatologist suggests that the qualitative conclusion of the method is sufficient.

Depth estimation from multispectral satellite imagery: a comparison of conventional and machine learning-based approaches- case study: Kish Island, Persian Gulf

Depth estimation from multispectral satellite imagery: a comparison of conventional and machine learning-based approaches- case study: Kish Island, Persian Gulf

Advanced Monocular Outdoor Pose Estimation in Autonomous Systems: Leveraging Optical Flow, Depth Estimation, and Semantic Segmentation with Dynamic Object Removal

Autonomous technologies have revolutionized transportation, military operations, and space exploration, necessitating precise localization in environments where traditional GPS-based systems are unreliable or unavailable. While widespread for outdoor localization, GPS systems face limitations in obstructed environments such as dense urban areas, forests, and indoor spaces. Moreover, GPS reliance introduces vulnerabilities to signal disruptions, which can lead to significant operational failures. Hence, developing alternative localization techniques that do not depend on external signals is essential, showing a critical need for robust, GPS-independent localization solutions adaptable to different applications, ranging from Earth-based autonomous vehicles to robotic missions on Mars. This paper addresses these challenges using Visual odometry (VO) to estimate a camera’s pose by analyzing captured image sequences in GPS-denied areas tailored for autonomous vehicles (AVs), where safety and real-time decision-making are paramount. Extensive research has been dedicated to pose estimation using LiDAR or stereo cameras, which, despite their accuracy, are constrained by weight, cost, and complexity. In contrast, monocular vision is practical and cost-effective, making it a popular choice for drones, cars, and autonomous vehicles. However, robust and reliable monocular pose estimation models remain underexplored. This research aims to fill this gap by developing a novel adaptive framework for outdoor pose estimation and safe navigation using enhanced visual odometry systems with monocular cameras, especially for applications where deploying additional sensors is not feasible due to cost or physical constraints. This framework is designed to be adaptable across different vehicles and platforms, ensuring accurate and reliable pose estimation. We integrate advanced control theory to provide safety guarantees for motion control, ensuring that the AV can react safely to the imminent hazards and unknown trajectories of nearby traffic agents. The focus is on creating an AI-driven model(s) that meets the performance standards of multi-sensor systems while leveraging the inherent advantages of monocular vision. This research uses state-of-the-art machine learning techniques to advance visual odometry’s technical capabilities and ensure its adaptability across different platforms, cameras, and environments. By merging cutting-edge visual odometry techniques with robust control theory, our approach enhances both the safety and performance of AVs in complex traffic situations, directly addressing the challenge of safe and adaptive navigation. Experimental results on the KITTI odometry dataset demonstrate a significant improvement in pose estimation accuracy, offering a cost-effective and robust solution for real-world applications.

FusionPortableV2: A unified multi-sensor dataset for generalized SLAM across diverse platforms and scalable environments

Simultaneous Localization and Mapping (SLAM) has been widely applied in various robotic missions, from rescue operations to autonomous driving. However, the generalization of SLAM algorithms remains a significant challenge, as current datasets often lack scalability in terms of platforms and environments. To address this limitation, we present FusionPortableV2, a multi-sensor SLAM dataset featuring sensor diversity, varied motion patterns, and a wide range of environmental scenarios. Our dataset comprises 27 sequences, spanning over 2.5 hours and collected from four distinct platforms: a handheld suite, a legged robot, an unmanned ground vehicle (UGV), and a vehicle. These sequences cover diverse settings, including buildings, campuses, and urban areas, with a total length of 38.7 km. Additionally, the dataset includes ground truth (GT) trajectories and RGB point cloud maps covering approximately 0.3 km2. To validate the utility of our dataset in advancing SLAM research, we assess several state-of-the-art (SOTA) SLAM algorithms. Furthermore, we demonstrate the dataset’s broad application beyond traditional SLAM tasks by investigating its potential for monocular depth estimation. The complete dataset, including sensor data, GT, and calibration details, is accessible at https://fusionportable.github.io/dataset/fusionportable_v2 .

DE-RGBD SLAM: Enhancing Static Feature Point Selection in RGB-D Visual SLAM Using Depth Information

Abstract Feature point extraction plays a key role in visual simultaneous localization and mapping (SLAM) systems. And it remains a major challenge to accurately select static feature points in a complex dynamic environment. To address this issue, this paper proposes an RGB-D SLAM method, referred to as DE-RGBD SLAM, which optimizes feature selection by integrating depth information and effectively utilizes depth data and multi-view geometric information to achieve localization and navigation for mobile robots in dynamic environments. Firstly, the method analyzes prominent feature regions in the image based on colour and depth information captured by an RGB-D camera. It sets adaptive FAST corner detection thresholds according to the grayscale information of these regions while masking other areas. Next, the method obtains in-depth information on the detected feature points in the current frame. It combines their pixel coordinates in the image coordinate system to determine the presence of redundant feature points. Notably, the method can detect some dynamic feature points between consecutive frames. Subsequently, in the camera coordinate system, the method compares the depth information of feature points in the depth image with the epipolar depth estimates derived from the essential matrix to determine whether the features are static and eliminate dynamic feature points. This approach significantly enhances the reliability of static feature points. Finally, the accuracy and robustness of the proposed method are validated through experiments conducted on the public TUM dataset and real-world scenarios compared to state-of-the-art visual SLAM systems.

Non-invasive in-situ monitoring of deep etching processes using terahertz metasurfaces

This study presents an in-situ and non-invasive process control and monitoring (PCM) method for deep silicon etching, leveraging terahertz metasurfaces. The technique addresses the challenges for monitoring deep and high aspect ratio etching processes, which are prevalent in semiconductor microfabrication. By incorporating metasurfaces with identical geometric shapes and sizes as crucial components of targeted devices, the method enables accurate monitoring of the etching depth in the process. Continuous shifts of terahertz reflection spectra provide information on etching depth, while abrupt change in the curves highlights the etching endpoint, preventing over-etching. For the commonly used comb-finger structure, numerical simulations demonstrate a strong linear relationship between etching depth and terahertz resonant wavelengths (nonlinearity < 1%) and an abrupt resonant frequency change (> 0.6 THz) at the endpoint. Experimental validations confirm the accuracy of the PCM method, with an etching depth estimation error below 2 µm. This approach enhances the precision of PCM in microfabrication, offering the potential for widespread applications in the production of micromechanical sensors, actuators, and other microelectronic devices.

Deep Neural Networks for Accurate Depth Estimation with Latent Space Features

Depth estimation plays a pivotal role in advancing human–robot interactions, especially in indoor environments where accurate 3D scene reconstruction is essential for tasks like navigation and object handling. Monocular depth estimation, which relies on a single RGB camera, offers a more affordable solution compared to traditional methods that use stereo cameras or LiDAR. However, despite recent progress, many monocular approaches struggle with accurately defining depth boundaries, leading to less precise reconstructions. In response to these challenges, this study introduces a novel depth estimation framework that leverages latent space features within a deep convolutional neural network to enhance the precision of monocular depth maps. The proposed model features dual encoder–decoder architecture, enabling both color-to-depth and depth-to-depth transformations. This structure allows for refined depth estimation through latent space encoding. To further improve the accuracy of depth boundaries and local features, a new loss function is introduced. This function combines latent loss with gradient loss, helping the model maintain the integrity of depth boundaries. The framework is thoroughly tested using the NYU Depth V2 dataset, where it sets a new benchmark, particularly excelling in complex indoor scenarios. The results clearly show that this approach effectively reduces depth ambiguities and blurring, making it a promising solution for applications in human–robot interaction and 3D scene reconstruction.

Monocular depth estimation based on deep learning for intraoperative guidance surface-enhanced Raman scattering (SERS) imaging

Monocular depth estimation based on deep learning for intraoperative guidance surface-enhanced Raman scattering (SERS) imaging

Estimating Water Depth of Different Waterbodies Using Deep Learning Super Resolution from HJ-2 Satellite Hyperspectral Images

Hyperspectral remote sensing images offer a unique opportunity to quickly monitor water depth, but how to utilize the enriched spectral information and improve its spatial resolution remains a challenge. We proposed a water depth estimation framework to improve spatial resolution using deep learning and four inversion methods and verified the effectiveness of different super resolution and inversion methods in three waterbodies based on HJ-2 hyperspectral images. Results indicated that it was feasible to use HJ-2 hyperspectral images with a higher spatial resolution via super resolution methods to estimate water depth. Deep learning improves the spatial resolution of hyperspectral images from 48 m to 24 m and shows less information loss with peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and spectral angle mapper (SAM) values of approximately 37, 0.92, and 2.42, respectively. Among four inversion methods, the multilayer perceptron demonstrates superior performance for the water reservoir, achieving the mean absolute error (MAE) and the mean absolute percentage error (MAPE) of 1.292 m and 22.188%, respectively. For two rivers, the random forest model proves to be the best model, with an MAE of 0.750 m and an MAPE of 10.806%. The proposed method can be used for water depth estimation of different water bodies and can improve the spatial resolution of water depth mapping, providing refined technical support for water environment management and protection.

Hydrocarbon Potential Evaluation of Chad Basin, Northeastern Nigeria Using High-Resolution Aeromagnetic Data

A cost-effective and non-intrusive aeromagnetic geophysical technique was utilized to reduce exploration uncertainty in the Chad Basin, which is believed to have significant hydrocarbon potential despite limited exploration due to its remote and challenging terrain. By analyzing various derivatives and magnetic attributes extracted from filtered and enhanced aeromagnetic data, the study aimed to highlight potential areas for hydrocarbon generation and preservation, allowing for more focused and detailed geophysical exploration efforts. This study produced several key maps, including a total magnetic intensity map (ranging from -168.7 to 224.8 nT), a reduction to equator map (ranging from -156.8 to 207.9 nT), and depth analysis, which helped delineate the structural framework and sedimentary thicknesses within the basin. Quantitative interpretation of these maps, along with the analytic depth map (ranging from 22.1 to 5649.3 m), tilt depth map (ranging from -76.50 to 80.40), and first and second vertical derivatives maps (ranging from -0.097 to 0.086 nT/m and -0.000240 to 0.000236 nT/m, respectively), was employed to detect geological features and thick sedimentary fills sufficient to create or trap hydrocarbons. Depth estimates indicate that the shallow sources range between 22 and 230 m, while deeper sources fall between 2747 and 6000 m, suggesting sufficient sedimentary fill for hydrocarbon generation. The basin is characterized by large sedimentary deposits that are bounded by structural features such as folds, faults, and fractures. The northern part of the basin, trending towards the northwest, eastern edge, and parts of the south, showed active structural features, including anticlines, synclines, and fractures, which are essential for petroleum trapping. These findings, coupled with the identified thicknesses and favorable structural disposition, suggest that the Chad Basin holds high potential for hydrocarbon accumulation, making it a prospective region for further exploration with more detailed geophysical methods.

Study on the Application Method of Aquifer Depth Distribution Patterns as Model Input on the Performance of a Physically Based Distributed Hydrologic Model

Groundwater discharge is critical for maintaining river flow during dry seasons, especially in lowland areas. Despite its significance, groundwater resources have often been overlooked highlighting the need for comprehensive studies amidst growing pressure to develop new water resources. This study focuses on the Soyang River Basin, South Korea, including its ungauged northern regions, the nearby DMZ (Demilitarized Zone), using the physically based Gridded Surface Subsurface Hydrologic Analysis (GSSHA) model. A three-year simulation was conducted to examine variable aquifer depth distribution patterns by assuming an inverse relationship between surface elevation and aquifer bottom depth. Three case studies (i.e., equal distribution, linear regression, and logarithmic regression) were evaluated and compared. The method to identity optimal aquifer depth distributions to enhance groundwater simulation accuracy in regions with significant topographical variation was incorporated. Groundwater levels at six monitoring sites showed that altitude-based variable aquifer depths outperformed the equal distribution case. The results showed strong agreement between simulated and observed values, particularly in the linear regression case with an R-squared statistic of 0.858 and Nash–Sutcliffe Efficiency index of 0.789, indicating that linear regression-based aquifer depth estimation can significantly improves long-term runoff modeling and groundwater simulation accuracy. The logarithmic regression case had the lowest relative peak error in peak flow. These findings highlight the importance of adjusting aquifer depth distributions in physically based hydrologic models to better reflect real-world conditions. Overall, this study contributes to advance groundwater modeling by integrating variable aquifer depth distributions into a physically based hydrologic model for large scale watersheds.

DyPanVO: a robust monocular visual odometry in dynamic environments by utilizing multiple deep neural networks

Abstract Traditional monocular Visual Odometry (VO) is typically based on the assumption of a static environment. However, it performs poorly in dynamic scenes, suffering from error accumulation and scale drift. To address these issues, a novel framework, named DyPanVO, was proposed, which incorporates multiple deep neural networks for feature point extraction, panoptic segmentation, and depth estimation. Firstly, learning-based methods are employed, specifically SuperPoint for robust feature extraction and LightGlue for accurate feature matching. Then, outlier points are eliminated by combining dynamic prior results from panoptic segmentation with epipolar geometry constraints to determine the motion state of objects. Additionally, the introduction of depth information ensures that scale is recovered and fundamentally solves the issue of error accumulation. Two types of experiment (outdoor and indoor scenes) are carried out to show the effectiveness of DyPanVO. Moreover, the comparison results demonstrate that it performs comparably with multi-view geometry-based and outperforms learning-based methods.

Reliable and Effective Stereo Matching for Underwater Scenes

Stereo matching plays a vital role in underwater environments, where accurate depth estimation is crucial for applications such as robotics and marine exploration. However, underwater imaging presents significant challenges, including noise, blurriness, and optical distortions that hinder effective stereo matching. This study develops two specialized stereo matching networks: UWNet and its lightweight counterpart, Fast-UWNet. UWNet utilizes self- and cross-attention mechanisms alongside an adaptive 1D-2D cross-search to enhance cost volume representation and refine disparity estimation through a cascaded update module, effectively addressing underwater imaging challenges. Due to the need for timely responses in underwater operations by robots and other devices, real-time processing speed is critical for task completion. Fast-UWNet addresses this challenge by prioritizing efficiency, eliminating the reliance on the time-consuming recurrent updates commonly used in traditional methods. Instead, it directly converts the cost volume into a set of disparity candidates and their associated confidence scores. Adaptive interpolation, guided by content and confidence information, refines the cost volume to produce the final accurate disparity. This streamlined approach achieves an impressive inference speed of 0.02 s per image. Comprehensive tests conducted in diverse underwater settings demonstrate the effectiveness of both networks, showcasing their ability to achieve reliable depth perception.

MSDFNet: multi-scale detail feature fusion encoder–decoder network for self-supervised monocular thermal image depth estimation

Abstract Currently available thermal image depth estimation methods are difficult to efficiently extract fine multi-scale feature information from thermal images and suffer from the problem of blurring details at the edges of the estimated depth map. To address these challenges, this paper proposes MSDFNet, a multi-scale detail feature fusion encoder–decoder network, for self-supervised monocular thermal image depth estimation. The model is based on a channel expansion hourglass residual lightweight feature encoder, which can capture rich and fine-grained multi-scale feature information with low computational effort. MSDFNet utilizes a detail feature weight evaluation decoder to fuse cross-scale features and reevaluate the importance of each feature, thereby emphasizing critical edge information at multiple scales. Additionally, MSDFNet incorporates a depth consistency loss function, which provides self-supervised signals for the detailed features of thermal images and improves the optimization of network performance. The method is applied to the ViViD++ and MS2 datasets and achieves state-of-the-art depth estimation performance compared to existing state-of-the-art algorithms. In the Indoor Dark scenario of the ViViD++ dataset, the Abs Rel, Sq Rel, RMSE, and RMSE log error metric values of MSDFNet are reduced by 6.71%, 11.92%, 9.09%, and 5.73%, respectively, while the accuracy metric values δ < 1.25 i , i = 1 , 2 , 3 were improved by 4.18%, 1.13%, and 0.2%, respectively. In addition, MSDFNet proves its excellent generalization ability on the MS2 dataset. The Abs Rel and RMSE error values in the night scene are reduced by 45.6% and 30.09%, respectively, and the accuracy δ < 1.25 i , i = 1 , 3 is improved by 20.95% and 1.33%, respectively. The Abs Rel and RMSE values in the rainy day scenario are reduced by 1.33% and 1.21%, respectively, and the accuracy δ < 1.25 i , i = 1 , 3 is improved by 0.24% and 0.83%, respectively.

Permafrost thaw subsidence, sea-level rise, and erosion are transforming Alaska’s Arctic coastal zone

Arctic shorelines are vulnerable to climate change impacts as sea level rises, permafrost thaws, storms intensify, and sea ice thins. Seventy-five years of aerial and satellite observations have established coastal erosion as an increasing Arctic hazard. However, other hazards at play-for instance, the cumulative impact that sea-level rise and permafrost thaw subsidence will have on permafrost shorelines-have received less attention, preventing assessments of these processes' impacts compared to and combined with coastal erosion. Alaska's Arctic Coastal Plain (ACP) is ideal for such assessments because of the high-density observations of topography, coastal retreat rates, and permafrost characteristics, and importance to Indigenous communities and oilfield infrastructure. Here, we produce 21st-century projections of Arctic shoreline position that include erosion, permafrost subsidence, and sea-level rise. Focusing on the ACP, we merge 5 m topography, satellite-derived coastal lake depth estimates, and empirical assessments of land subsidence due to permafrost thaw with projections of coastal erosion and sea-level rise for medium and high emissions scenarios from the Intergovernmental Panel on Climate Change's AR6 Report. We find that by 2100, erosion and inundation will together transform the ACP, leading to 6-8x more land loss than coastal erosion alone and disturbing 8-11x more organic carbon. Without mitigating measures, by 2100, coastal change could damage 40 to 65% of infrastructure in present-day ACP coastal villages and 10 to 20% of oilfield infrastructure. Our findings highlight the risks that compounding climate hazards pose to coastal communities and underscore the need for adaptive planning for Arctic coastlines in the 21st century.

Diffraction model-driven neural network with semi-supervised training strategy for real-world 3D holographic photography

Compared to traditional 2D displays, 3D display technology provides richer information to the viewer. Learning-based computer-generated holography (CGH) has shown great potential in realizing real-time holographic 3D displays. However, most of the current learning-based CGH algorithms cannot quickly complete the training stage and produce high-quality holograms due to insufficient constraints in the training stage of the neural network. In this paper, we propose a diffractive model-driven neural network trained using a semi-supervised training (SST-holo) strategy and incorporate a state-of-the-art monocular depth estimation algorithm to achieve the fast generation of holograms of real-world 3D scenes. Compared to the supervised training strategy, our proposed semi-supervised training strategy does not require high-quality labeled datasets, but can significantly improve the imaging quality and generalization of the algorithm. Incorporating the Res-MSR block in SST-holo to adaptively learn image features of different scales enhances the learning capability of the network. In addition, we adopt a random splicing processing strategy to preprocess the dataset to ensure that the original features in the dataset are not corrupted. SST-holo can generate high-quality 3D phase-only holograms with 2 K resolution in 0.015 seconds. Both monochrome and color optical experiments show that the proposed algorithm has good 3D effect and generalization ability and can effectively improve the quality of reconstructed images.

Secrets of Event-Based Optical Flow, Depth and Ego-Motion Estimation by Contrast Maximization.

Event cameras respond to scene dynamics and provide signals naturally suitable for motion estimation with advantages, such as high dynamic range. The emerging field of event-based vision motivates a revisit of fundamental computer vision tasks related to motion, such as optical flow and depth estimation. However, state-of-the-art event-based optical flow methods tend to originate in frame-based deep-learning methods, which require several adaptations (data conversion, loss function, etc.) as they have very different properties. We develop a principled method to extend the Contrast Maximization framework to estimate dense optical flow, depth, and ego-motion from events alone. The proposed method sensibly models the space-time properties of event data and tackles the event alignment problem. It designs the objective function to prevent overfitting, deals better with occlusions, and improves convergence using a multi-scale approach. With these key elements, our method ranks first among unsupervised methods on the MVSEC benchmark and is competitive on the DSEC benchmark. Moreover, it allows us to simultaneously estimate dense depth and ego-motion, exposes the limitations of current flow benchmarks, and produces remarkable results when it is transferred to unsupervised learning settings. Along with various downstream applications shown, we hope the proposed method becomes a cornerstone on event-based motion-related tasks.

Weakly-Supervised Depth Estimation and Image Deblurring via Dual-Pixel Sensors.

Dual-pixel (DP) imaging sensors are getting more popularly adopted by modern cameras. A DP camera captures a pair of images in a single snapshot by splitting each pixel in half. Several previous studies show how to recover depth information by treating the DP pair as an approximate stereo pair. However, dual-pixel disparity occurs only in image regions with defocus blur which is unlike classic stereo disparity. Heavy defocus blur in DP pairs affects the performance of depth estimation approaches based on matching. Therefore, we treat the blur removal and the depth estimation as a joint problem. We investigate the formation of the DP pair, which links the blur and depth information, rather than blindly removing the blur effect. We propose a mathematical DP model that can improve depth estimation by the blur. This exploration motivated us to propose our previous work, an end-to-end DDDNet (DP-based Depth and Deblur Network), which jointly estimates depth and restores the image in a supervised fashion. However, collecting the ground-truth (GT) depth map for the DP pair is challenging and limits the depth estimation potential of the DP sensor. Therefore, we propose an extension of the DDDNet, called WDDNet (Weakly-supervised Depth and Deblur Network), which includes an efficient reblur solver that does not require GT depth maps for training. To achieve this, we convert all-in-focus images into supervisory signals for unsupervised depth estimation in our WDDNet. We jointly estimate an all-in-focus image and a disparity map, then use a Reblur and Fstack module to regularize the disparity estimation and image restoration. We conducted extensive experiments on synthetic and real data to demonstrate the competitive performance of our method when compared to state-of-the-art (SOTA) supervised approaches.

Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-Shot Metric Depth and Surface Normal Estimation.

We introduce Metric3D v2, a geometric foundation model designed for zero-shot metric depth and surface normal estimation from single images, critical for accurate 3D recovery. Depth and normal estimation, though complementary, present distinct challenges. State-of-the-art monocular depth methods achieve zero-shot generalization through affine-invariant depths, but fail to recover real-world metric scale. Conversely, current normal estimation techniques struggle with zero-shot performance due to insufficient labeled data. We propose targeted solutions for both metric depth and normal estimation. For metric depth, we present a canonical camera space transformation module that resolves metric ambiguity across various camera models and large-scale datasets, which can be easily integrated into existing monocular models. For surface normal estimation, we introduce a joint depth-normal optimization module that leverages diverse data from metric depth, allowing normal estimators to improve beyond traditional labels. Our model, trained on over 16 million images from thousands of camera models with varied annotations, excels in zero-shot generalization to new camera settings. As shown in Fig. 1, It ranks the 1st in multiple zero-shot and standard benchmarks for metric depth and surface normal prediction. Our method enables the accurate recovery of metric 3D structures on randomly collected internet images, paving the way for plausible single-image metrology. Our model also relieves the scale drift issues of monocular-SLAM (Fig. 3), leading to high-quality metric scale dense mapping. Such applications highlight the versatility of Metric3D v2 models as geometric foundation models.