Outdoor Images Research Articles

Sand dust image visibility enhancement algorithm via fusion strategy

The outdoor images captured in sand dust weather often suffer from poor contrast and color distortion, which seriously interfere with the performance of intelligent information processing systems. To solve the issues, a novel enhancement algorithm based on fusion strategy is proposed in this paper. It includes two components in sequence: sand removal via the improved Gaussian model-based color correction algorithm and dust elimination using the residual-based convolutional neural network (CNN). Theoretical analysis and experimental results show that compared with the prior sand dust image enhancement methods, the proposed fusion strategy can effectively correct the overall yellowing hue and remove the dust haze disturbance, which provides a constructive idea for the future development of sand dust image enhancement.

Trends and Prospects of Techniques for Haze Removal From Degraded Images: A Survey

For the last two decades, image processing techniques have been used frequently in computer vision applications. The most challenging task in image processing is restoring images that are degraded due to various weather conditions. Mainly, the visibility of outdoor images is corrupted due to adverse atmospheric effects. The visibility of acquired images is reduced in these circumstances. Haze is an atmospheric phenomenon that reduces the clarity of an image. Due to the presence of particles such as dust, dirt, soot, or smoke, there is significant decay in the color and contrast of captured images. Haze present in acquired images causes issues in a variety of computer vision applications. Therefore, enhancing the contrast of a hazy image and restoring the visibility of the scene is essential. Since clear images are required in every application, image dehazing is an important step. Hence, many researchers are working on it. Different methods have been presented in the literature for image dehazing. This study describes various traditional and deep learning techniques of image dehazing from an analytical perspective. The main intention behind this work is to provide an intuitive understanding of the major techniques that have made a relevant contribution to haze removal. In this paper, we have covered different types of contributions toward dehazing based on the traditional method as well as deep learning approaches. With a considerable amount of instinctive simplifications, the reader is expected to have an improved ability to visualize the internal dynamics of these processes.

Efficiently Guided Active Contours for Image Segmentation

Segmenting natural and outdoor images are challenging for most of the latest variational segmentation models. For this purpose we employ derived image data (DID) and propose a robust variational model. The DID rely on three images by utilizing image local and global statistics as well as filter image which is obtained through our design high pass filtering techniques. Then these derived image data are incorporated into our proposed energy functional which can robustly segment images having inhomogeneity, mix backgrounds and multi-regions. Furthermore, the results of DID are compared with other well known methods with finding Jaccard similarity index to proof the efficient and qualitative performance of proposed model over the traditional methods. Finally, the proposed DID based model is tested on real world 3D images to ensure that it also preserve its performance in vector valued images as well.

Illuminant segmentation for multi-illuminant scenes using latent illumination encoding

Color constancy is an important part of the Human Visual System that allows us to recognize colors of object invariant to the light that is illuminating them. Computational color constancy is the process of estimating the illumination of a scene using some computational method. However, this problem is inherently ill-posed. The RGB value of each pixel is dependent on spectral reflectance of the object and the spectral power distribution of the illumination. Hence, methods that try to solve the computational color constancy problem have to introduce assumptions about the illumination. One common assumption is that there is only one global source of illumination, i.e., that the illumination is constant across the whole scene. Under this assumption, modern color constancy methods achieve excellent results, usually predicting the illumination color with accuracy better than the human eye. This assumption is broken in many real-world multi-illuminant scenes, e.g., outdoor images where parts of the scene are illuminated by either sunlight or skylight. This leads to significant drop in accuracy of single-illuminant estimation methods. Therefore, in this work, we propose a novel method for segmenting images based on illumination in multi-illuminant scenes. This method detects regions where there is only one illuminant, thus detecting areas where the single-illuminant assumption holds. We show that our method produces excellent results and outperforms all baseline models by a large margin.

불균일 안개 영상 합성을 이용한 딥러닝 기반 안개 영상 깊이 추정

Image depth estimation is a technology that is the basis of various image analysis. As analysis methods using deep learning models emerge, studies using deep learning in image depth estimation are being actively conducted. Currently, most deep learning-based depth estimation models are being trained with clean and ideal images. However, due to the lack of data on adverse conditions such as haze or fog, the depth estimation may not work well in such an environment. It is hard to sufficiently secure an image in these environments, and in particular, obtaining non-homogeneous haze data is a very difficult problem. In order to solve this problem, in this study, we propose a method of synthesizing non-homogeneous haze images and a learning method for a monocular depth estimation deep learning model using this method. Considering that haze mainly occurs outdoors, datasets mainly containing outdoor images are constructed. Experiment results show that the model with the proposed method is good at estimating depth in both synthesized and real haze data.

WCATN: Unsupervised deep learning to classify weather conditions from outdoor images

Weather classification from single images is significant for many outdoor computer vision applications, while it has not been thoroughly studied. Existing methods view it as a supervised task under the guidance of weather labels. Though these methods have made great success, they are not applicable to real-world applications because of their reliance on massive human-annotated weather images. In this paper, we take the initial attempt to treat weather classification as an unsupervised task, i.e. to classify weather conditions from unlabeled outdoor images. Specifically, we propose a three-step unsupervised approach to automatically group images into weather clusters. In the first step, i.e. weather feature learning, semantically meaningful weather features are learned by employing a self-supervised task. Furthermore, we also introduce online triplet mining into the self-supervised task to extract discriminative weather features. In the second step, i.e. weather clustering, we design a learnable clustering method via mining the nearest neighbors and enforcing consistency between each sample and its nearest neighbors. To alleviate the uncertainty of weather clustering due to noisy nearest neighbors, we propose a self-labeling method as the third step to employ the already well-classified samples for fine-tuning. Experimental results on two publicly available weather image datasets demonstrate that the proposed approach achieves promising performance.

OutCast: Outdoor Single‐image Relighting with Cast Shadows

AbstractWe propose a relighting method for outdoor images. Our method mainly focuses on predicting cast shadows in arbitrary novel lighting directions from a single image while also accounting for shading and global effects such the sun light color and clouds. Previous solutions for this problem rely on reconstructing occluder geometry, e. g., using multi‐view stereo, which requires many images of the scene. Instead, in this work we make use of a noisy off‐the‐shelf single‐image depth map estimation as a source of geometry. Whilst this can be a good guide for some lighting effects, the resulting depth map quality is insufficient for directly ray‐tracing the shadows. Addressing this, we propose a learned image space ray‐marching layer that converts the approximate depth map into a deep 3D representation that is fused into occlusion queries using a learned traversal. Our proposed method achieves, for the first time, state‐of‐the‐art relighting results, with only a single image as input. For supplementary material visit our project page at: dgriffiths.uk/outcast.

A Fish Retina-Inspired Single Image Dehazing Method

Outdoor images are significantly degraded by bad weather conditions, such as fog and dust, which seriously limits efficient information extraction by computer vision applications. Most existing weather-degraded hazy image enhancement methods ignore the wavelength dependence of the scattering coefficient and therefore cannot well handle the colorized haze in which the medium transmission varies in different color channels. In this work, we propose a fish retina-inspired method to solve this problem. Fish have special retinal mechanisms for extracting useful information from underwater environments where photons are scattered and absorbed spatially introducing wavelength-dependent degradation of visibility. Due to the similarity between suspensions in water and aerosols in the air, these mechanisms should also be suitable for processing hazy scenes. The proposed method imitates bipolar cells with scene-dependent center-surround receptive fields to eliminate redundant information, ganglion cells with nonlinear processing to boost contrast, and the centrifugal pathway from amacrine cells to the horizontal cells to adaptively protect visual information from excessive lateral suppression. Compared with state-of-the-art methods on various images degraded by fog, haze, or dust, the proposed method shows quite competitive performance both qualitatively and quantitatively.

A single image dehazing method based on decomposition strategy

Outdoor haze has adverse impact on outdoor image quality, including contrast loss and poor visibility. In this paper, a novel dehazing algorithm based on the decomposition strategy is proposed. It combines the advantages of the two-dimensional variational mode decomposition (2DVMD) algorithm and dark channel prior. The original hazy image is adaptively decomposed into low-frequency and high-frequency images according to the image frequency band by using the 2DVMD algorithm. The low-frequency image is dehazed by using the improved dark channel prior, and then fused with the high-frequency image. Furthermore, we optimize the atmospheric light and transmittance estimation method to obtain a defogging effect with richer details and stronger contrast. The proposed algorithm is compared with the existing advanced algorithms. Experiment results show that the proposed algorithm has better performance in comparison with the state-of-the-art algorithms.

A CNN-based multi-task framework for weather recognition with multi-scale weather cues

Automatic image-based weather recognition has a great significance in the practical use of computer visual applications. The weather recognition approaches via multi-task learning obtain the description of weather conditions by jointly tackling weather-cue segmentation task and weather classification task. However, the existing approaches neglect the multiple scales of weather cues, which may decrease the ability to deal with both large and small weather cues in weather-cue segmentation. They also fail to assign appropriate weights, which the multi-task learning for weather recognition relies on, to the two tasks. To address these problems, we propose a novel end-to-end Convolutional Neural Network (CNN)-based Multi-task Weather Recognition Network with multi-scale weather cues (CMWRN). Specifically, in the weather-cue segmentation task, we propose to segment weather cues, e.g. blue sky and black clouds, by capturing multi-scale weather-cue features. Arbitrary-scale weather-cue regions in the outdoor images can be accurately and effectively classified via resampling the weather-cue features at multiple scales. In the weather classification task, a generalized pooling method is introduced to generate more discriminative weather representations. To further promote the weather classification, the multi-scale weather-cue features are exploited to provide weather clues for weather representations. Moreover, we develop an adaptive weighting scheme to automatically balance the weather classification task and weather-cue segmentation task for more effective training. Compared with the state-of-the-arts, experimental results on two public available benchmark datasets demonstrate the superior performance of our approach.

Outdoor Image Weather Recognition Based on Image Blocks and Feature Fusion

Outdoor Image Weather Recognition Based on Image Blocks and Feature Fusion

Remove and recover: Deep end-to-end two-stage attention network for single-shot heavy rain removal

In this paper, we propose a novel heavy rain removal algorithm using a deep neural network. Unlike most of the existing deraining methods, heavy rain removal is a more challenging task because it is necessary to remove both the rain marks and the haze effects, which are entangled in a complex manner. Motivated by this, we propose a new end-to-end two-stage attention network for single-shot heavy rain removal. The proposed network is connected serially with a removal network and a recovery network, which are based on a newly introduced dilation-wise attention block and skip attention block. Based on these attention techniques, the removal network predicts the heavy rain effect that needs to be removed from a given image, and the recovery network successfully predicts the details that need to be recovered, resulting in a clean image. We also introduce a new realistic RainCityscapes+ dataset, composed of synthesized outdoor images, and demonstrate extensive experiments, the results of which show our approach outperforms the state-of-the-art methods on both real and synthetic datasets quantitatively and qualitatively.

Can keypoint descriptors be used in outdoor image classification?

In this paper, we approach a Content Based Image Retrieval problem using keypoints and texture. We employ an algorithm that allows computing a fixed number of keypoints for all the images in the dataset, by adapting a certain parameter of the keypoints computation method. The SIFT (Scale Invariant Feature Transform) and SURF (Speeded Up Robust Features) algorithms yield the required keypoints. We also used texture features to improve the retrieval results. The texture features are computed using Local Binary Patterns (LBP), Gray Level Co-occurrence Matrices (GLCM), Dual Tree Complex Wavelet Transform (DTCWT), and Gabor filters. The methods were tested on our own dataset, ODIDS which contains outdoor images, mainly acquired in the city of Iaşi, Romania. We performed location identification with good results.

Robust region-based active contour models via local statistical similarity and local similarity factor for intensity inhomogeneity and high noise image segmentation

<p style='text-indent:20px;'>In this paper, we design a novel variational segmentation method for two types of segmentation problems, namely, global segmentation (all objects /features in a given image are aimed to be segmented) and selective/ interactive segmentation (an objects /feature of interest in a given image is aimed to be segmented) for inhomogeneous and severe additive noisy images. The proposed segmentation models implement a local denoising constraint, capable to tackle efficiently noise/outliers and coping with intensity inhomogeneity issues, combined with local similarity factor based on spatial distances and intensity differences in the local region that guides accurately the level set function to distinguish between outliers and minute important details. Furthermore, to exhibit the accuracy of the proposed models, an experimental comparison is inducted and shown comparisons with state-of-art models on synthetic images, outdoor images, and medical images.</p>

Content-Aware Detection of Temporal Metadata Manipulation

Most pictures shared online are accompanied by temporal metadata (i.e., the day and time they were taken), which makes it possible to associate an image content with real-world events. Maliciously manipulating this metadata can convey a distorted version of reality. In this work, we present the emerging problem of detecting timestamp manipulation. We propose an end-to-end approach to verify whether the purported time of capture of an outdoor image is consistent with its content and geographic location. We consider manipulations done in the hour and/or month of capture of a photograph. The central idea is the use of supervised consistency verification, in which we predict the probability that the image content, capture time, and geographical location are consistent. We also include a pair of auxiliary tasks, which can be used to explain the network decision. Our approach improves upon previous work on a large benchmark dataset, increasing the classification accuracy from 59.0% to 81.1%. We perform an ablation study that highlights the importance of various components of the method, showing what types of tampering are detectable using our approach. Finally, we demonstrate how the proposed method can be employed to estimate a possible time-of-capture in scenarios in which the timestamp is missing from the metadata.

Single Image Raindrop Removal Using a Non-Local Operator and Feature Maps in the Frequency Domain

Taking a photo on a rainy day may result in a photo with raindrops. Images containing raindrops have a significant impact on the visual impression and accuracy when applied to image recognition systems. Thus, an automatic high-quality raindrop removal method is desired for outdoor image processing systems as well as for acquiring good-looking images. Several existing methods have been proposed to tackle this problem, but they often fail to keep global consistency and generate unnatural patterns. In this paper, we tackle this problem by introducing a non-local operator. The non-local operator combines features in distant locations with matrix multiplication and enables consistency in distant locations. In addition, high-frequency components such as edges are more affected in images with raindrops. Inspired by the nature that high-frequency components can be separated from other components in the frequency domain, we also propose to process feature maps in the frequency domain, which are obtained by the fast Fourier transform operation and processed by several convolution layers. Experimental results show that our method effectively removes raindrops and achieves state-of-the-art performance.

A Unified Approach for Representing Outdoor Scenes

Generalized image understanding of natural outdoor images is fundamentally a bottom-up approach guided by the data and information received from the digital input images. The importance of intermediate image representation that aids generalized image understanding, inspired the proposed a unified approach for providing a new meaningful and objective representation of a given outdoor scene. The proposed unified framework provides a novel set of string and semantic codes, based on inferential statistical data obtained from segments characterization and blocks characterization of natural outdoor images. The framework acts as an analytical tool that can categorize the input images into four types based on the inferential statistical representation - 1) Object-based image, 2) Heterogeneous color based texture image, 3) Homogeneous color based texture image, 4) Pure toned image. The proposed string and semantic code characterization is also rotation and scale variant. Such a representation is simple, unsupervised, computationally inexpensive and also places minimal demand on the storage. Later for a higher-level image analysis and understanding the proposed unified approach can provide focused interaction between the above intermediate representation and domain scene knowledge for object detection, classification, or recognition.

Multiscale Image Dehazing and Restoration: An Application for Visual Surveillance

The captured outdoor images and videos may appear blurred due to haze, fog, and bad weather conditions. Water droplets or dust particles in the atmosphere cause the light to scatter, resulting in very limited scene discernibility and deterioration in the quality of the image captured. Currently, image dehazing has gained much popularity because of its usability in a wide variety of applications. Various algorithms have been proposed to solve this ill-posed problem. These algorithms provide quite promising results in some cases, but they include undesirable artifacts and noise in haze patches in adverse cases. Some of these techniques take unrealistic processing time for high image resolution. In this paper, to achieve real-time halo-free dehazing, fast and effective single image dehazing we propose a simple but effective image restoration technique using multiple patches. It will improve the shortcomings of DCP and improve its speed and efficiency for high-resolution images. A coarse transmission map is estimated by using the minimum of different size patches. Then a cascaded fast guided filter is used to refine the transmission map. We introduce an efficient scaling technique for transmission map estimation, which gives an advantage of very low-performance degradation for a high-resolution image. For performance evaluation, quantitative, qualitative and computational time comparisons have been performed, which provide quiet faithful results in speed, quality, and reliability of handling bright surfaces.

Fast outdoor hazy image dehazing based on saturation and brightness

Haze usually limits the visibility and reduces the contrast of outdoor images. The removal of haze in a single image has always been a challenging problem. Recently, many methods have been proposed to effectively remove haze in harsh conditions. However, these methods fail to balance the dehazing effect and time consumption. This paper proposes a method based on HSV colour space to restore the visibility of uniform scattering medium. It can prevent the atmospheric light and transmission from being miscalculated. This method uses the brightness component of haze image to estimate the global atmospheric light, which reduces the influence of luminescent objects on atmospheric light estimation. Then, this paper deduces the estimation model of the saturation of scene radiance based on the atmospheric scattering model. In the meantime, according to the advantages and problems of the model, the fast estimation of the transmission is realized by using the stretching function. Finally, this paper solves the parameters in the model by iterative method. Since this method can estimate different media transmission for each pixel, better results can be obtained. The simulation results show that the algorithm is superior to the existing algorithms in terms of dehazing effect and time consumption.

Brightness-gradient difference feature guided shadow removal method

Shadow removal is a fundamental and pivotal task to build the high-level cognition in the computer vision field. Due to the fact that the existing shadow removal methods cannot effectively remove shadows from the outdoor image and deal with shadow boundaries, we construct a convolutional neural network without the process of shadow detection for the shadow removal task. The constructed CNN avoids the risk of gradient vanishing by designing double-attention residual block and improves the performance of shadow removal by fusing the knowledge transfer idea. Specially, we design a hand-crafted feature, named brightness-gradient difference feature, to distinguish shadow boundary pixels from non-shadow boundary pixels, and the designed feature is fused into the loss function to dilute or even eliminate the existing shadow boundaries. Extensive experiments using three public shadow removal benchmarks with three measurable indicators are reported in this paper. The results of experiments demonstrate that the proposed method has an effective performance for the shadow removal task. The ablation studies validate the structural rationality of the proposed method.