Single Outdoor Image Research Articles

Outdoor illumination estimation via all convolutional neural networks

When inserting a virtual object into an outdoor image, the recovery of sun orientation has a critical effect on the fusion of virtual objects and real scenes. For the problem of sun orientation estimation from a single outdoor image, a new end-to-end learning method is proposed in this paper. A luminance channel of sky region is introduced into the input to enhance the extracted image features. A network only consists of convolutional layers is designed to improve the ability of extracting image features. Pruning and quantization are used to compress the network, resulting in a large reduction in the number of network parameters and the storage space, only with a slight loss of precision.

Detecting Shadows from a Single Outdoor Image Based on High Order Energy Function

Detecting Shadows from a Single Outdoor Image Based on High Order Energy Function

Sun orientation estimation from a single image using short-cuts in DCNN

The illumination effect is essential for the realistic results in images which are created by inserting virtual objects into real scene. For outdoor scenes, automatic estimation of sun orientation condition from a single outdoor image is fundamental for inserting 3D models to a single image. Traditional methods for outdoor sun orientation estimation often use handcraft illumination features or cues. These cues heavily rely on the experiences of human and pre-processing progresses using other image understanding technologies such as shadow and sky detection, geometry recovery and intrinsic image decomposition, which limit their performances. We propose an end to end way of outdoor sun orientation estimation via a novel deep convolutional neural network (DCNN), which directly outputs the orientation of the sun from an outdoor image. Our proposed SunOriNet contains a contact layer that directly contacts the intermediate feature maps to the high-level ones and learns hierarchical features automatically from a large-scale image dataset with annotated sun orientations. The experiments reveal that our DCNN can well estimate sun orientation from a single outdoor image. The estimation accuracy of our method outperforms state-of-the-art DCNN based methods.

Two-Class Weather Classification.

Given a single outdoor image, we propose a collaborative learning approach using novel weather features to label the image as either sunny or cloudy. Though limited, this two-class classification problem is by no means trivial given the great variety of outdoor images captured by different cameras where the images may have been edited after capture. Our overall weather feature combines the data-driven convolutional neural network (CNN) feature and well-chosen weather-specific features. They work collaboratively within a unified optimization framework that is aware of the presence (or absence) of a given weather cue during learning and classification. In this paper we propose a new data augmentation scheme to substantially enrich the training data, which is used to train a latent SVM framework to make our solution insensitive to global intensity transfer. Extensive experiments are performed to verify our method. Compared with our previous work and the sole use of a CNN classifier, this paper improves the accuracy up to 7-8 percent. Our weather image dataset is available together with the executable of our classifier.

Pixel-wise orthogonal decomposition for color illumination invariant and shadow-free image.

In this paper, we propose a novel, effective and fast method to obtain a color illumination invariant and shadow-free image from a single outdoor image. Different from state-of-the-art methods for shadow-free image that either need shadow detection or statistical learning, we set up a linear equation set for each pixel value vector based on physically-based shadow invariants, deduce a pixel-wise orthogonal decomposition for its solutions, and then get an illumination invariant vector for each pixel value vector on an image. The illumination invariant vector is the unique particular solution of the linear equation set, which is orthogonal to its free solutions. With this illumination invariant vector and Lab color space, we propose an algorithm to generate a shadow-free image which well preserves the texture and color information of the original image. A series of experiments on a diverse set of outdoor images and the comparisons with the state-of-the-art methods validate our method.

Shadow Detection Using Multi-Features in SVM Classifier

Shadows may cause many problems in computer vision, such as object recognition, image segmentation and video surveillance. In this paper, we present a new method to detect cast shadow in a single outdoor image. We build up an illumination model to explain the process of shadow formed, and through this model we introduce some useful features. The regions for extract features are acquired through canny edge detector, after a series of morphological operations. Then we use SVM classifier with a multi-kernel model to train these features for shadow region classification. Our results show that edges of shadow images can be detected effectively with our methods.

Detecting soft shadows in a single outdoor image: From local edge-based models to global constraints

Detecting soft shadows from a single image has been a difficult problem because soft shadows usually retain the textures of their backgrounds and because pixel intensities are greatly variable. In this paper, we propose a novel approach to automatically detect soft shadow edges from a single outdoor photograph. We first propose an edge-based model of a soft shadow boundary and parameterize three features. Then, motivated by the observation that the intensity profiles along the normal direction of the hard shadow edge can be approximated by a Sigmoid function, we develop an intensity model including the intensity, intensity change rate and intensity variance of the soft shadow. By fitting the intensity model of the soft shadows, three features of the boundary model are obtained and the initial segments of the soft shadow boundary are localized. Finally, to generate a continuous boundary, a level set-based optimization process is utilized with a global constraint on shadow width. Experiments demonstrate the effectiveness and flexibility of our method.

View Transformation Based on a Single Outdoor Image

Abstract An automatic approach for view transformation based on a single outdoor image is proposed in this paper. First, the hierarchical segmentation method is conducted to segment an outdoor image into several meaningful regions and each region is labelled as sky, ground or standing object. Then, different methods are used to estimate each region's depth according to its label. After that, the obtained depth information is utilized to create a new view image after any rotation, translation and pitch. Finally, the image inpainting work for the missing colour region is accomplished using its neighbour's colour. Extensive experiments show the proposed approach not only improves the accuracy of view transformation, but also performs well even for images with occlusion phenomena.

Depth estimation and occlusion boundary recovery from a single outdoor image

A novel depth estimation and occlusion boundary recovery approach for a single outdoor image is described. This work is distinguished by three contributions. The first contribution is the introduction of a new depth estimation model, which takes the camera rotation and pitch into account, thus improving the depth estimation accuracy. The second contribution is a depth estimation algorithm, in which we classify the standing object region with visible ground-contact points into three cases according to the information of vanishing point for the first time, meanwhile, we propose the depth reference line concept for estimating the depth of the region with depth change. Two advantages can thereby be obtained: improving the depth estimation accuracy further and avoiding the occlusion mismarked phenomenon. The third contribution is the depth estimation method for the standing object region without visible ground-contact points, which takes the mean of minimum and maximum depth estimation result as region depth and prevents the missing phenomenon of occlusion boundaries. Extensive experiments show that our works are better than previously published results.

Estimating the Natural Illumination Conditions from a Single Outdoor Image

Given a single outdoor image, we present a method for estimating the likely illumination conditions of the scene. In particular, we compute the probability distribution over the sun position and visibility. The method relies on a combination of weak cues that can be extracted from different portions of the image: the sky, the vertical surfaces, the ground, and the convex objects in the image. While no single cue can reliably estimate illumination by itself, each one can reinforce the others to yield a more robust estimate. This is combined with a data-driven prior computed over a dataset of 6 million photos. We present quantitative results on a webcam dataset with annotated sun positions, as well as quantitative and qualitative results on consumer-grade photographs downloaded from Internet. Based on the estimated illumination, we show how to realistically insert synthetic 3-D objects into the scene, and how to transfer appearance across images while keeping the illumination consistent.

Tricolor Attenuation Model for Shadow Detection

Shadows, the common phenomena in most outdoor scenes, bring many problems in image processing and computer vision. In this paper, we present a novel method focusing on extracting shadows from a single outdoor image. The proposed tricolor attenuation model (TAM) that describe the attenuation relationship between shadow and its nonshadow background is derived based on image formation theory. The parameters of the TAM are fixed by using the spectral power distribution (SPD) of daylight and skylight, which are estimated according to Planck's blackbody irradiance law. Based on the TAM, a multistep shadow detection algorithm is proposed to extract shadows. Compared with previous methods, the algorithm can be applied to process single images gotten in real complex scenes without prior knowledge. The experimental results validate the performance of the model.

Retinex theory‐based shadow detection and removal in single outdoor image

PurposeShadows, the common phenomena in most outdoor scenes, bring many problems in practical image processing. Shadow detection and removal, especial in uncalibrated outdoor image, is still a difficult problem. The purpose of this paper is to detect and to remove shadows in single outdoor image based on retinex theory.Design/methodology/approachThe shadow extraction algorithm originates from a simple idea that the human‐vision‐based retinex has the natural ability to enhance the shadow regions of an image no matter it is penumbrae or umbrae. Shadows are detected by comparing the retinex‐enhanced images with original images in the paper. The shadow removal algorithm in the paper deals with the shadow regions and non‐shadow regions in the images separately using the retinex enhancement algorithm. Through adding smooth light forcibly to shadow edges and introducing shadow edge masks, the authors reduce the effects of shadow edges in shadow removal processing.FindingsSome real single outdoor images with the umbra regions and those with penumbra regions are both experimented in the paper. Experimental results validate the feasibility of the approach.Originality/valueThe approach proposed here does not use any special prior knowledge and assumptions. The feasibility of this method is testified for detecting and removing both penumbrae and umbrae in the outdoor images.