Scene Depth Research Articles

Multi-scale depth information fusion network for image dehazing

According to the atmospheric physical model, we can use accurate transmittance and atmospheric light information to convert a hazy image into a clean one. The scene-depth information is very important for image dehazing due to the transmittance directly corresponds to the scene depth. In this paper, we propose a multi-scale depth information fusion network based on the U-Net architecture. The model uses hazy images as inputs and extracts the depth information from these images; then, it encodes and decodes this information. In this process, hazy image features of different scales are skip-connected to the corresponding positions. Finally, the model outputs a clean image. The proposed method does not rely on atmospheric physical models, and it directly outputs clean images in an end-to-end manner. Through numerous experiments, we prove that the multi-scale deep information fusion network can effectively remove haze from images; it outperforms other methods in the synthetic dataset experiments and also performs well in the real-scene test set.

Range-intensity-profile prior dehazing method for underwater range-gated imaging

This paper is concerned with the mitigation of backscatter effects in a single gated image. A range-intensity-profile prior dehazing method is proposed to estimate scene depth and finely remove water backscatter at different depths for underwater range-gated imaging. It is based on the prior that the target intensity is distributed with range intensity profiles in gated images. The depth transmission and depth-noise map are then calculated from the scene depth. A high-quality image is restored by subtracting the depth-noise map and dividing the depth transmission. The simulation and experimental results show that the proposed method works well even if a portion of the estimated depth may be smaller than its real value, and the peak signal-to-noise ratio of dehazing images gets up to a doubled increase.

Single Color Image De-Hazing Using Dark Channel Prior

In real world scenario due to bad weather conditions the presence of fog and haze, the particles in the outdoor environment or atmosphere (e.g., droplets, smoke, sand, snow, mist, volcanic ash, liquid dust or solid dust) greatly reduces the visibility of the scene. As a consequence, the clarity of an image would be seriously degraded, which may decrease the performance of many image processing applications. Image Dehazing methods try to alleviate these problems by estimating a haze free version of the given hazy image. Traditionally the task of image dehazing can be processed as recovering the scene radiance from a noisy hazy image by estimating the atmospheric light and transmission properties. In those kinds of techniques, it additionally needs some more information regarding the image such as scene depth, weather condition parameters and so on. But this is not suitable for real world scenario. This research focus on proposing an approach to fully capture the intrinsic attributes of a hazy image and improves the performance of dehazing. Dark Channel Prior plays vital role in dehazing process. Hence this research focus on recovering dehaze version of the input image by CNN. So that all methods are comes under the categories image enhancement, image fusion image restoration based on statistical and structural features of the hazed image.

Perceptual Monocular Depth Estimation

Monocular depth estimation (MDE), which is the task of using a single image to predict scene depths, has gained considerable interest, in large part owing to the popularity of applying deep learning methods to solve “computer vision problems”. Monocular cues provide sufficient data for humans to instantaneously extract an understanding of scene geometries and relative depths, which is evidence of both the processing power of the human visual system and the predictive power of the monocular data. However, developing computational models to predict depth from monocular images remains challenging. Hand-designed MDE features do not perform particularly well, and even current “deep” models are still evolving. Here we propose a novel approach that uses perceptually-relevant natural scene statistics (NSS) features to predict depths from monocular images in a simple, scale-agnostic way that is competitive with state-of-the-art systems. While the statistics of natural photographic images have been successfully used in a variety of image and video processing, analysis, and quality assessment tasks, they have never been applied in a predictive end-to-end deep-learning model for monocular depth. Correspondingly, no previous work has explicitly incorporated perceptual features in a monocular depth-prediction approach. Here we accomplish this by developing a new closed-form bivariate model of image luminances and use features extracted from this model and from other NSS models to drive a novel deep learning framework for predicting depth given a single image.

Exploring Image Enhancement for Salient Object Detection in Low Light Images

Low light images captured in a non-uniform illumination environment usually are degraded with the scene depth and the corresponding environment lights. This degradation results in severe object information loss in the degraded image modality, which makes the salient object detection more challenging due to low contrast property and artificial light influence. However, existing salient object detection models are developed based on the assumption that the images are captured under a sufficient brightness environment, which is impractical in real-world scenarios. In this work, we propose an image enhancement approach to facilitate the salient object detection in low light images. The proposed model directly embeds the physical lighting model into the deep neural network to describe the degradation of low light images, in which the environment light is treated as a point-wise variate and changes with local content. Moreover, a Non-Local-Block Layer is utilized to capture the difference of local content of an object against its local neighborhood favoring regions. To quantitative evaluation, we construct a low light Images dataset with pixel-level human-labeled ground-truth annotations and report promising results on four public datasets and our benchmark dataset.

Stereoscopic Image Retargeting Based on Deep Convolutional Neural Network

Stereoscopic image retargeting aims at converting stereoscopic images to the target resolution adaptively. Different from 2D image retargeting, stereoscopic image retargeting needs to preserve both the shape structure of salient objects and depth consistency of 3D scenes. In this paper, we present a stereoscopic image retargeting method based on deep convolutional neural network to obtain high-quality retargeted images with both object shape preservation and scene depth preservation. First, a cross-attention extraction mechanism is constructed to generate attention map, which contains the valuable attention features of the left and right images and the common attention features between them. Second, since the disparity map can provide accurate depth information of objects in 3D scenes, a disparity-assisted 3D significance map generation module is utilized to further preserve the valuable depth information of stereoscopic images. Finally, in order to predict the retargeted stereoscopic images accurately, an image consistency loss is developed to preserve the geometric structure of salient objects, and a disparity consistency loss is introduced to eliminate depth distortions. Experimental results demonstrate that the proposed deep convolutional neural network can provide favorable stereoscopic image retargeting results.

Suspect Multifocus Image Fusion Based on Sparse Denoising Autoencoder Neural Network for Police Multimodal Big Data Analysis

In recent years, the success rate of solving major criminal cases through big data has been greatly improved. The analysis of multimodal big data plays a key role in the detection of suspects. However, the traditional multiexposure image fusion methods have low efficiency and are largely time-consuming due to the artifact effect in the image edge and other sensitive factors. Therefore, this paper focuses on the suspect multiexposure image fusion. The self-coding neural network based on deep learning has become a hotspot in the research of data dimension reduction, which can effectively eliminate the irrelevant and redundant learning data. In the case of limited field depth, due to the limited focusing depth of the camera, the focusing plane cannot obtain the global clear image of the target in the depth scene, which is prone to defocusing and blurring phenomena. Therefore, this paper proposes a multifocus image fusion based on a sparse denoising autoencoder neural network. To realize an unsupervised end-to-end fusion network, the sparse denoising autoencoder neural network is adopted to extract features and learn fusion rules and reconstruction rules simultaneously. The initial decision graph of the multifocus image is taken as a prior input to learn the rich detailed information of the image. The local strategy is added to the loss function to ensure that the image is restored accurately. The results show that this method is superior to the state-of-the-art fusion methods.

WITHDRAWN: Application of CNN and gated recurrent network for visual improvement of dehazing

Abstract Various restoration model depending on the prior knowledge had consider an arduous vision task. We present in this paper an effective algorithm for restoring a recognizable picture from a hazy image. Instead of using conventional methods i.e.; image priors as the restoration process, in form of color attenuation, dark channels, or color line contrast, we use a point-to-point gated background aggregation structure to reestablish the final image free from haze. A hybrid dehazing algorithm of CNN framework with features learning recurrent modeling is applied. The transmission map or its depth of scene, structural attributes is obtained in form of ranking layer is applied with the CNN network model so that hazy images can be captured at same instant. Post processing is carried out by Fast guided filter. Hence this hybrid model basically gives a inter connection idea between coarse layer and fine layer by extensive results obtained on experimental, simulated and real-world images prove that the proposed algorithm operates against state-of-the-art algorithms in a beneficial way.

IDRLP: Image Dehazing Using Region Line Prior.

In this work, a novel and ultra-robust single image dehazing method called IDRLP is proposed. It is observed that when an image is divided into n regions, with each region having a similar scene depth, the brightness of both the hazy image and its haze-free correspondence are positively related with the scene depth. Based on this observation, this work determines that the hazy input and its haze-free correspondence exhibit a quasi-linear relationship after performing this region segmentation, which is named as region line prior (RLP). By combining RLP and the atmospheric scattering model (ASM), a recovery formula (RF) can be easily obtained with only two unknown parameters, i.e., the slope of the linear function and the atmospheric light. A 2D joint optimization function considering two constraints is then designed to seek the solution of RF. Unlike other comparable works, this "joint optimization" strategy makes efficient use of the information across the entire image, leading to more accurate results with ultra-high robustness. Finally, a guided filter is introduced in RF to eliminate the adverse interference caused by the region segmentation. The proposed RLP and IDRLP are evaluated from various perspectives and compared with related state-of-the-art techniques. Extensive analysis verifies the superiority of IDRLP over state-of-the-art image dehazing techniques in terms of both the recovery quality and efficiency. A software release is available at https://sites.google.com/site/renwenqi888/.

Accuracy in Depth Recovery and 3D Image Synthesis From Single Image Using Multi-Color Filter Aperture and Shallow Depth of Field

A computational 3D image generation using a single view with multi-color filter aperture (MCA) and multi-plane representation is a cost-effective approach and most useful when there is no option to acquire either stereo or multi-views with orientation at all. Although this approach generates 3D perception image that includes multiple objects with both similar and dissimilar colors having occluded by each other, it may be insufficient for virtual/augmented reality applications due to inaccurate depth. In this article, we obtain a more accurate geometric depth estimation by formulating a suitable relationship between inter-objects depth of the 3D scene in the depth-of-field (DoF) zone and its corresponding inter-image plane depths of a 3D perception image in depth-of-focus (DoFo) zone of a given camera under shallow DoF zone constraint. But, this shallow depth zone is configured to be dependent only on the focal distance between the lens and object while the remaining parameters such as aperture diameter, focal length, and sensor sensitivity are held at constant values. All-in-focus 3D perception image is synthesized from multi-plane images (MPIs) by utilizing the inter-image plane depths computed from the disparities caused across the boundaries and its smooth surface from image textures inside the respective boundaries of the 2D MCA image. The 2.1D sketch is used as a semantic segmentation technique to determine the number of objects in the 3D scene as one in-focus region and the rest as out-of-focus regions due to the circle of confusions (CoCs) on the fixed image sensor plane. The same enables both ordering of the image regions and identifying occlusion wherever applicable. An accurate depth 3D image is synthesized, replacing accurate inter-depths in place of inter-depth between MPIs used for 3D perception image. In the end, the paper summarizes few experimental validations for the proposed approach with some salient examples having depth gaps between 0.5cm to 10.5cm.

Single-Image Real-Time Rain Removal Based on Depth-Guided Non-Local Features.

Rain is a common weather phenomenon that affects environmental monitoring and surveillance systems. According to an established rain model (Garg and Nayar, 2007), the scene visibility in the rain varies with the depth from the camera, where objects faraway are visually blocked more by the fog than by the rain streaks. However, existing datasets and methods for rain removal ignore these physical properties, thus limiting the rain removal efficiency on real photos. In this work, we analyze the visual effects of rain subject to scene depth and formulate a rain imaging model that collectively considers rain streaks and fog. Also, we prepare a dataset called RainCityscapes on real outdoor photos. Furthermore, we design a novel real-time end-to-end deep neural network, for which we train to learn the depth-guided non-local features and to regress a residual map to produce a rain-free output image. We performed various experiments to visually and quantitatively compare our method with several state-of-the-art methods to show its superiority over others.

Depth Estimation from a Single Image Based on Cauchy Distribution Model

Most approaches to estimate a scene’s 3D depth from a single image often model the point spread function (PSF) as a 2D Gaussian function. However, those methods are suffered from some noises, and difficult to get a high quality of depth recovery. We presented a simple yet effective approach to estimate exactly the amount of spatially varying defocus blur at edges, based on a Cauchy distribution model for the PSF. The raw image was re-blurred twice using two known Cauchy distribution kernels, and the defocus blur amount at edges could be derived from the gradient ratio between the two re-blurred images. By propagating the blur amount at edge locations to the entire image using the matting interpolation, a full depth map was then recovered. Experimental results on several real images demonstrated both feasibility and effectiveness of our method, being a non-Gaussian model for DSF, in providing a better estimation of the defocus map from a single un-calibrated defocused image. These results also showed that our method was robust to image noises, inaccurate edge location and interferences of neighboring edges. It could generate more accurate scene depth maps than the most of existing methods using a Gaussian based DSF model.

Unsupervised Monocular Depth Estimation via Recursive Stereo Distillation.

Existing unsupervised monocular depth estimation methods resort to stereo image pairs instead of ground-truth depth maps as supervision to predict scene depth. Constrained by the type of monocular input in testing phase, they fail to fully exploit the stereo information through the network during training, leading to the unsatisfactory performance of depth estimation. Therefore, we propose a novel architecture which consists of a monocular network (Mono-Net) that infers depth maps from monocular inputs, and a stereo network (Stereo-Net) that further excavates the stereo information by taking stereo pairs as input. During training, the sophisticated Stereo-Net guides the learning of Mono-Net and devotes to enhance the performance of Mono-Net without changing its network structure and increasing its computational burden. Thus, monocular depth estimation with superior performance and fast runtime can be achieved in testing phase by only using the lightweight Mono-Net. For the proposed framework, our core idea lies in: 1) how to design the Stereo-Net so that it can accurately estimate depth maps by fully exploiting the stereo information; 2) how to use the sophisticated Stereo-Net to improve the performance of Mono-Net. To this end, we propose a recursive estimation and refinement strategy for Stereo-Net to boost its performance of depth estimation. Meanwhile, a multi-space knowledge distillation scheme is designed to help Mono-Net amalgamate the knowledge and master the expertise from Stereo-Net in a multi-scale fashion. Experiments demonstrate that our method achieves the superior performance of monocular depth estimation in comparison with other state-of-the-art methods.

A Semiphysical Approach of Haze Removal for Landsat Image

The presence of haze could seriously contaminate the observations of optical satellite imagery. Haze not only significantly affects the visual interpretation but also reduces the accuracy of map products. In this article, a semiphysical approach is proposed to reduce the haze effects for Landsat image. The proposed approach is based on the physical model of radiative transfer theory and the presence of dark objects. As the depth map of satellite remotely sensed image is almost a constant value, the coarse transmission map of atmosphere is estimated by the haze thickness, other than the scene depth map. The derived coarse transmission is utilized to correct the color shift induced by airlight. For haze veiled textural information, the guided-filter based approach is adopted to refine the coarse transmission map to restore the textural information. Experiments are conducted upon the images acquired by Landsat at different dates and spatial locations. The visual interpretation upon the dehazed results suggests that the proposed approach could generate visually promising results and preserve spectral properties of land surfaces. Moreover, it also performs favorably against several state-of-the-art deep learning based methods and a classic algorithm. The results of quantitative assessments demonstrate that the developed approach could enhance the information of Landsat scene with minimum spectrum changes. With the visual and quantitative assessments, we can conclude that the proposed approach is a promising solution for Landsat image dehazing. It is expected to improve the data quality of hazy scene and expand the usability of Landsat data.

Fusion Key Frame Image Confidence Assessment of the Medical Service Robot Whole Scene Reconstruction

Facing the precise service and emergency rescue needs of medical service robots in irregular scene, in order to achieve better navigation and path planning for robots in service scenarios, for the whole reconstruction of the absolute scale service scenario, this article proposes a frame of whole scene three-dimensional (3D) point cloud reconstruction based on the fusion of scene depth estimation, confidence assessment, and pose tracking with monocular camera. The algorithm first collects the scene focus stack images under an initial viewing angle through the robot mobile terminal of camera. The absolute depth information of the scene is estimated on the server side, and the confidence level of the reconstructed image of the point cloud is evaluated, and non- uniform sampling is performed to reduce the influence of the error estimation. Based on the sparse key frame position information defined by monocular SLAM, the 3D reconstruction of the whole scene in absolute scale is realized through multi-perspective point cloud pose matching. It provides information of cloud reconstruction of scenic spots for target recognition and navigation of a medical service robot.

An Effective Algorithm for Specular Reflection Image Enhancement

Specular highlights often exist in real-world images due to the material property of objects and the capturing environments. High light in real-life scenes can reduce image quality and cause loss of image information, to overcome this, we propose an effective specular reflection image enhancement algorithm. First, the scene depth map of specular reflection image was obtained by the color attenuation prior method; then, two adaptive adjustment factors were developed to optimize the transmission map for specular reflection images and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{0}$ </tex-math></inline-formula> gradient minimization filter was used to eliminate halo artifacts. Finally, the nonuniform illumination compensation strategy based on the YUV color space was used to expand the dynamic range of the image and increase the saturation of the image. Experimental results via subjective and objective evaluations demonstrate that the proposed method can provide better visual quality than other methods and can effectively enhance the information in the specular reflection image.

RS-DPSNet: Deep Plane Sweep Network for Rolling Shutter Stereo Images

Since the rolling shutter (RS) camera successively exposes each scanline, accurately reconstructing scene depth from an RS stereo image pair remains a great challenge. Directly applying the deep-learning-based depth estimation methods tailored for the global shutter (GS) stereo images leads to undesirable RS depth results due to inherent flaws in the network structure. In this letter, we fill this gap by developing an end-to-end RS-stereo-aware plane sweep network to improve the accuracy of the classic GS-based algorithm ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> DPSNet) in estimating the RS depth map. Specifically, we derive the RS-stereo-aware plane sweep model and further produce a more accurate and efficient cost volume through the effective incorporation of this model within DPSNet. Furthermore, to enable learning-based approaches to address the depth estimation problem in the context of RS stereo images, we contribute the first RS stereo dataset, CARLA-RSS. Experimental results demonstrate that our proposed pipeline achieves state-of-the-art performance.

Virtual Viewpoint Film and Television Synthesis Based on the Intelligent Algorithm of Wireless Network Communication for Image Repair

With the development of the computer vision field, the acquisition of scene depth information is one of the important topics in the three‐dimensional reconstruction of the computer vision field, and its significance is particularly important. The purpose of this paper is to study the virtual viewpoint video synthesis for image restoration based on the intelligent algorithm of wireless network communication. Aiming at the hole problem caused by the change of occlusion relationship, this paper proposes a hole‐filling method based on background recognition. A threshold segmentation algorithm is used to reduce the filling priority of foreground pixels at the boundary of the hole and fully solve the hole problem. This paper also proposes a wireless sensor network node positioning model with swarm intelligence algorithm, which combines swarm intelligence algorithm with some key issues of wireless sensor network, speeds up the convergence, and improves the traditional intelligence algorithm. According to the experimental data in this paper, the algorithm in this paper is about 20% higher than the traditional algorithm in PSNR. On SSIM, the performance of the algorithm in this paper is 4.6% higher than the traditional algorithm at most, and the lowest is 2.2%.

PotNet: Pothole detection for autonomous vehicle system using convolutional neural network

Advancement in vision-based techniques has enabled the autonomous vehicle system (AVS) to understand the driving scene in depth. The capability of autonomous vehicle system to understand the scene, and detecting the specific object depends on the strong feature representation of such objects. However, pothole objects are difficult to identify due to their non-uniform structure in challenging, and dynamic road environments. Existing approaches have shown limited performance for the precise detection of potholes. The study on the detection of potholes, and intelligent driving behaviour of autonomous vehicle system is little explored in existing articles. Hence, here, an improved prototype model, which is not only truly capable of detecting the potholes but also shows its intelligent driving behaviour when any pothole is detected, is proposed. The prototype is developed using a convolutional neural network with a vision camera to explore, and validates the potential, and autonomy of its driving behaviour in the prepared road environment. The experimental analysis of the proposed model on various performance measures have obtained accuracy, sensitivity, and F-measure of 99.02%, 99.03%, and 98.33%, respectively, which are comparable with the available state-of-art techniques.

A comparative study of single image fog removal methods

The presence of fog degrades visibility in natural scene conditions. Computer vision applications like navigation, tracking, and surveillance need clear atmospheric images or videos as prerequisites for optimal performance. However, foggy atmosphere creates problems for computer vision applications due to reduced visibility. Different fog removal techniques are used to improve the visual quality of images and videos. The fog density depends on the depth information. Scene depth information estimation needs multiple images, which limits its real-life application. Hence, a single image fog removal requires some prior knowledge and/or assumptions to get the depth information. In this paper, the recent fog removal techniques are grouped into three broad categories: (1) filter-based methods, (2) color correction based methods, and (3) learning-based methods, for ease of understanding. The primary objective is to provide an introduction to this field and compare performance (both qualitative and quantitative) of representative techniques for each category. It is found that filter-based methods are doing overall better compared to other categories.