Moving Cast Shadows Research Articles

A Comprehensive Review of Vehicle Detection Techniques Under Varying Moving Cast Shadow Conditions Using Computer Vision and Deep Learning

Design of a vision-based traffic analytic system for urban traffic video scenes has a great potential in context of Intelligent Transportation System (ITS). It offers useful traffic-related insights at much lower costs compared to their conventional sensor based counterparts. However, it remains a challenging problem till today due to the complexity factors such as camera hardware constraints, camera movement, object occlusion, object speed, object resolution, traffic flow density, and lighting conditions etc. ITS has many applications including and not just limited to queue estimation, speed detection and different anomalies detection etc. All of these applications are primarily dependent on sensing vehicle presence to form some basis for analysis. Moving cast shadows of vehicles is one of the major problems that affects the vehicle detection as it can cause detection and tracking inaccuracies. Therefore, it is exceedingly important to distinguish dynamic objects from their moving cast shadows for accurate vehicle detection and recognition. This paper provides an in-depth comparative analysis of different traffic paradigm-focused conventional and state-of-the-art shadow detection and removal algorithms. Till date, there has been only one brief survey which highlights the shadow removal methodologies particularly for traffic paradigm. In this paper, a number of research papers containing results of urban traffic scenes have been shortlisted from the last three decades to give a comprehensive overview of the work done in this area. The study reveals that the only way to make a comparative evaluation is to use the existing Highway I, II, and III datasets which are frequently used for qualitative or quantitative analysis of shadow detection or removal algorithms. Furthermore, the paper not only provides cues to solve moving cast shadow problems, but also suggests that even after the advent of Convolutional Neural Network (CNN)-based vehicle detection methods, the problems caused by moving cast shadows persists. Therefore, this paper proposes a hybrid approach which uses a combination of conventional and state-of-the-art techniques as a pre-processing step for shadow detection and removal before using CNN for vehicles detection. The results indicate a significant improvement in vehicle detection accuracies after using the proposed approach.

An Effective Framework Using Region Merging and Learning Machine for Shadow Detection and Removal

Moving cast shadows of moving objects significantly degrade the performance of many high-level computer vision applications such as object tracking, object classification, behavior recognition and scene interpretation. Because they possess similar motion characteristics with their objects, moving cast shadow detection is still challenging. In this paper, the foreground is detected by background subtraction and the shadow is detected by combination of Mean-Shift and Region Merging Segmentation. Using Gabor method, we obtain the moving targets with texture features. According to the characteristics of shadow in HSV space and texture feature, the shadow is detected and removed to eliminate the shadow interference for the subsequent processing of moving targets.
 Finally, to guarantee the integrity of shadows and objects for further image processing, a simple post-processing procedure is designed to refine the results, which also drastically improves the accuracy of moving shadow detection. Extensive experiments on publicly common datasets that the performance of the proposed framework is superior to representative state-of-the-art methods.

Temporal integration in the perception and discrimination of solid shape.

Two experiments evaluated the importance of temporal integration for the perception and discrimination of solid object shape. In Experiment 1, observers anorthoscopically viewed moving or stationary cast shadows of naturally shaped solid objects (bell peppers, Capsicum annuum) through narrow (4-mm wide) slits. At any given moment, observers could only see a very small portion of the overall object shape (generally less than 10%). The results showed that the observers' discrimination performance for the moving cast shadows was much higher than that obtained for the stationary shadows, demonstrating the ability to temporally integrate the piecemeal momentary information about shape that was available through the narrow apertures. In a second experiment, estimates of the strength of the observers' impressions of solid shapes rotating in depth were obtained as well as discrimination accuracies; perceptions of the original moving condition were compared with a new condition where the frames of the apparent motion sequences depicting solid objects in continuous motion (behind the slits) were randomly scrambled. The observers perceived the anorthoscopic displays as depicting solid objects rotating in depth, but only in the continuous motion condition. Interestingly, the discrimination performance in the scrambled condition remained relatively high-observers were still able to integrate information across the multiple scrambled frames in order to produce discrimination performance that was significantly higher than that obtained in the stationary shadow condition. This study was the first to thoroughly evaluate whether and to what extent human observers can effectively discriminate and perceive solid object shape anorthoscopically.

Moving Cast Shadows Segmentation Using Illumination Invariant Feature

This paper presents an effective framework for removing moving cast shadows. Taking the reflection property of object surface for shadow regions under static and fixed scenes, an approximation estimation strategy of bidirectional reflectance distribution function as illumination invariant feature is proposed. It is valid for different types of shadow scenes. In this paper, we propose a new multiple ratios-based technique to justify shadow type for each frame: intensity ratio, area ratio and edge ratio of shadow regions are introduced. According to shadow types, several specified strategies are designed. For weak shadows, multiple features fusion strategy is employed, including color constancy, texture consistency and illumination invariant. For strong shadows, illumination invariant is utilized to detect the umbra and color constancy is utilized to detect the penumbra. Moreover, a suite of shadow direction features is firstly proposed to identify penumbra. The proposed approach is verified in fourteen video sequences varying from weak to strong shadows. The experimental results demonstrate the effectiveness and robustness of the proposed method for both indoor and outdoor scenes compared with some state-of-the-art approaches.

An Effective Framework Using Spatial Correlation and Extreme Learning Machine for Moving Cast Shadow Detection

Moving cast shadows of moving objects significantly degrade the performance of many high-level computer vision applications such as object tracking, object classification, behavior recognition and scene interpretation. Because they possess similar motion characteristics with their objects, moving cast shadow detection is still challenging. In this paper, we present a novel moving cast-shadow detection framework based on the extreme learning machine (ELM) to efficiently distinguish shadow points from the foreground object. First, according to the physical model of shadows, pixel-level features of different channels in different color spaces and region-level features derived from the spatial correlation of neighboring pixels are extracted from the foreground. Second, an ELM-based classification model is developed by labelled shadow and un-shadow points, which is able to rapidly distinguish the points in the new input whether they belong to shadows or not. Finally, to guarantee the integrity of shadows and objects for further image processing, a simple post-processing procedure is designed to refine the results, which also drastically improves the accuracy of moving shadow detection. Extensive experiments on two publicly common datasets including 13 different scenes demonstrate that the performance of the proposed framework is superior to representative state-of-the-art methods.

Moving cast shadow detection using scale-relation multi-layer pooling features

Moving cast shadows detection and removal are indispensable for object detection and are the problems in visual surveillance applications which have been studied over the years. However, finding an efficient model that can handle the issue of moving cast shadow in various situations is still challenging. Unlike prior methods, we use a data-driven method without strong parametric assumptions or complex models to address the problem of moving cast shadow. In this paper, we propose a novel feature-extracting framework called Scale-Relation Multi-Layer Pooling Feature Extracting (SMPF) which includes two main tasks: (1) Scale-Relation Scheme (SRS), (2) Multi-Layer Pooling Scheme (MLPS). By leveraging the scale space, SRS firstly decomposes feature images of each shadow properties into various scales and further considers the relationship between adjacent scaled feature images of each shadow properties to extract the scale-relation features. Then, we design the multi-layer pooling scheme (MLPS) to integrate the features in a local region and to reduce the dimension of extracting features. After that, the density map is generated for various properties of shadow with low dimension. Finally, to seek the criteria for discriminating moving cast shadow, we use random forest algorithm as the ensemble decision scheme. The main contributions of this study are (1) we design the features with multi-scale which can provide abundant information to describe the moving cast shadow, (2) the multi-layer pooling scheme generates the density map to integrate and reduce the dimensions of features. Experiments on the popular benchmarks and the proposed dataset with benchmarks demonstrate that the proposed method can achieve the performances of the popular methodologies.

Novel Approach for Detection and Removal of Moving Cast Shadows Based on RGB, HSV and YUV Color Spaces

Cast shadow affects computer vision tasks such as image segmentation, object detection and tracking since objects and shadows share the same visual motion characteristics. This unavoidable problem decreases video surveillance system performance. The basic idea of this paper is to exploit the evidence that shadows darken the surface which they are cast upon. For this reason, we propose a simple and accurate method for detection of moving cast shadows based on chromatic properties in RGB, HSV and YUV color spaces. The method requires no a priori assumptions regarding the scene or lighting source. Starting from a normalization step, we apply canny filter to detect the boundary between self-shadow and cast shadow. This treatment is devoted only for the first sequence. Then, we separate between background and moving objects using an improved version of Gaussian mixture model. In order to remove these unwanted shadows completely, we use three change estimators calculated according to the intensity ratio in HSV color space, chromaticity properties in RGB color space, and brightness ratio in YUV color space. Only pixels that satisfy threshold of the three estimators are labeled as shadow and will be removed. Experiments carried out on various video databases prove that the proposed system is robust and efficient and can precisely remove shadows for a wide class of environment and without any assumptions. Experimental results also show that our approach outperforms existing methods and can run in real-time systems.

Perception of motion trajectories of objects from moving cast shadows in human (Homo sapiens) infants, Japanese macaque (Macaca fuscata) infants, and an adult chimpanzee (Pan troglodytes)(Summary of Awarded Presentations at the 24th Annual Meeting)

Perception of motion trajectories of objects from moving cast shadows in human (Homo sapiens) infants, Japanese macaque (Macaca fuscata) infants, and an adult chimpanzee (Pan troglodytes)(Summary of Awarded Presentations at the 24th Annual Meeting)

Unobtrusive silhouette extraction using multivariate analysis and shadow removal in RGB color model

Object extraction is the initial step for any image processing task, such as visual surveillance, object tracking and intelligent transport system. Apart from other techniques, such as biometric system, these methods do not require any physical contact with an individual and can be performed from a distance and, therefore, used severely. Although several object extraction methods are available, this paper proposes a simple method to extract an object from video sequence based on the correlation between color model components. As the use of a single threshold is not suitable to correctly segment the object from the background, we are using two thresholds. Experimental results prove that this method can work well in static as well as dynamic background having different illumination conditions. Moreover, segmentation has side effects like moving cast shadows and self shadow, both of which reduce accuracy. In this paper, we present an automatic shadow removal method to extract objects. The analytical comparison with other foreground extraction, shadow detection and removal algorithms and their results are also presented for better understanding. The proposed method extract silhouette without any pre or post processing step of shadow removal.

Real-time detection of moving cast shadows using foreground luminance statistics

This paper presents a robust real-time method for detection of moving cast shadows which employs the assumption of higher interdependence of luminance values for the shadow pixels in larger regions compared to the object pixels. First, a fast modified image differencing technique is used to separate foreground pixels from the background. Next, for a moving window of fixed width scanning the foreground regions, a new measure called Modified Correlation is introduced. The new measure is determined by first computing the correlation between the luminance values of the moving window and luminance values of its neighbouring windows; this correlation is then divided by a robust-to-noise range measured based on the luminance values of the moving window. The modified correlation exhibits abrupt rising transitions as it enters the shadow region from the object region, transitions which can be used to separate object pixels from shadow pixels. Thus, the new method is very effective at suppressing moving cast shadows, while avoiding limiting structures, unrealistic assumptions, the need for a-priori knowledge, and manual selection of critical parameters. An average shadow detection rate of 85.4% and an average shadow discrimination rate of 99.5% over multiple different sequences, higher than those of available methods in the literature, confirm the efficacy of the method. The robustness of the method is examined under different lighting conditions, different target-environment combinations, and sequences with object-shadow occlusion. The proposed method is computationally efficient and suitable for real-time situations.

Multiview Intrinsic Images of Outdoors Scenes with an Application to Relighting

We introduce a method to compute intrinsic images for a multiview set of outdoor photos with cast shadows, taken under the same lighting. We use an automatic 3D reconstruction from these photos and the sun direction as input and decompose each image into reflectance and shading layers, despite the inaccuracies and missing data of the 3D model. Our approach is based on two key ideas. First, we progressively improve the accuracy of the parameters of our image formation model by performing iterative estimation and combining 3D lighting simulation with 2D image optimization methods. Second, we use the image formation model to express reflectance as a function of discrete visibility values for shadow and light, which allows to introduce a robust visibility classifier for pairs of points in a scene. This classifier is used for shadow labeling, allowing to compute high-quality reflectance and shading layers. Our multiview intrinsic decomposition is of sufficient quality to allow relighting of the input images. We create shadow-caster geometry which preserves shadow silhouettes and, using the intrinsic layers, we can perform multiview relighting with moving cast shadows. We present results on several multiview datasets, and show how it is now possible to perform image-based rendering with changing illumination conditions.

Moving Cast Shadow Detection

Motion perception is an amazing innate ability of the creatures on the planet. This adroitness entails a functional advantage that enables species to compete better in the wild. The motion perception ability is usually employed at different levels, allowing from the simplest interaction with the ’physis’ up to the most transcendental survival tasks. Among the five classical perception system , vision is the most widely used in the motion perception field. Millions years of evolution have led to a highly specialized visual system in humans, which is characterized by a tremendous accuracy as well as an extraordinary robustness. Although humans and an immense diversity of species can distinguish moving object with a seeming simplicity, it has proven to be a difficult and non trivial problem from a computational perspective.In the field of Computer Vision, the detection of moving objects is a challenging and fundamental research area. This can be referred to as the ’origin’ of vast and numerous vision-based research sub-areas. Nevertheless, from the bottom to the top of this hierarchical analysis, the foundations still relies on when and where motion has occurred in an image.Pixels corresponding to moving objects in image sequences can be identified by measuring changes in their values. However, a pixel’s value (representing a combination of color and brightness) could also vary due to other factors such as: variation in scene illumination, camera noise and nonlinear sensor responses among others. The challenge lies in detecting if the changes in pixels’ value are caused by a genuine object movement or not. An additional challenging aspect in motion detection is represented by moving cast shadows. The paradox arises because a moving object and its cast shadow share similar motion patterns. However, a moving cast shadow is not a moving object. In fact, a shadow represents a photometric illumination effect caused by the relative position of the object with respect to the light sources.Shadow detection methods are mainly divided in two domains depending on the application field. One normally consists of static images where shadows are casted by static objects, whereas the second one is referred to image sequences where shadows are casted by moving objects. For the first case, shadows can provide additional geometric and semantic cues about shape and position of its casting object as well as the localization of the light source. Although the previous information can be extracted from static images as well as video sequences, the main focus in the second area is usually change detection, scene matching or surveillance. In this context, a shadow can severely affect with the analysis and interpretation of the scene.The work done in the thesis is focused on the second case, thus it addresses the problem of detection and removal of moving cast shadows in video sequences in order to enhance the detection of moving object.

Perception of the motion trajectory of objects from moving cast shadows in infant Japanese macaques (Macaca fuscata)

The shadows cast by moving objects enable human adults and infants to infer the motion trajectories of objects. Nonhuman animals must also be able to discriminate between objects and their shadows and infer the spatial layout of objects from cast shadows. However, the evolutionary and comparative developmental origins of sensitivity to cast shadows have not been investigated. In this study, we used a familiarity/novelty preferential looking procedure to assess the ability of infant macaques, aged 7-24 weeks, to discriminate between a 'depth' display containing a ball and cast shadow moving diagonally and an 'up' display containing a ball with a diagonal trajectory and a shadow with a horizontal trajectory. The infant macaques could discriminate the trajectories of the balls based on the moving shadows. These findings suggest that the ability to perceive the motion trajectory of an object from the moving shadow is common to both humans and macaques.

Integrated moving cast shadows detection method for surveillance videos

Moving cast shadow detection plays an exceedingly important role in content-based analysis in surveillance videos. A novel integrated moving cast shadow detection method is proposed for accurate moving objects detection. Based on the analysis for shadow model, we first exploit color information sufficiently in hue, saturation, and value color space and texture similarity in texture space to separate moving cast shadows from the foreground, respectively. Then we obtain the rough shadow detection result by means of synthesizing the above two results efficiently. Finally, post processing is put forward to modify misclassified pixels for acquiring the refined shadow detection result. Extensive experiments and comparisons of various scenes prove that the proposed method can detect cast shadows automatically and effectively and outperforms some well-known methods.

Visual Sensor Based Abnormal Event Detection with Moving Shadow Removal in Home Healthcare Applications

Vision-based abnormal event detection for home healthcare systems can be greatly improved using visual sensor-based techniques able to detect, track and recognize objects in the scene. However, in moving object detection and tracking processes, moving cast shadows can be misclassified as part of objects or moving objects. Shadow removal is an essential step for developing video surveillance systems. The goal of the primary is to design novel computer vision techniques that can extract objects more accurately and discriminate between abnormal and normal activities. To improve the accuracy of object detection and tracking, our proposed shadow removal algorithm is employed. Abnormal event detection based on visual sensor by using shape features variation and 3-D trajectory is presented to overcome the low fall detection rate. The experimental results showed that the success rate of detecting abnormal events was 97% with a false positive rate of 2%. Our proposed algorithm can allow distinguishing diverse fall activities such as forward falls, backward falls, and falling asides from normal activities.

Moving shadow detection method based on PCA

Moving cast shadows generally exist in video sequence.To prevent moving shadows being misclassified as moving objects or parts of moving objects,and improve the accuracy of shadows detection algorithm,an algorithm of moving cast shadows was proposed based on spatial information.First,an adaptive background was generated by building Gaussian mixture background model,and the feature was extracted using Principal Component Analysis(PCA) based on transformation.Then,the moving cast shadows were detected utilizing the space coordination of shadow and object pixels.Experimental results show that the proposed algorithm is robust to noise,and can relieve the influence of illumination change and detect moving shadows correctly.

Real-Time Moving Vehicle Detection With Cast Shadow Removal in Video Based on Conditional Random Field

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> This paper presents an approach of moving vehicle detection and cast shadow removal for video based traffic monitoring. Based on conditional random field, spatial and temporal dependencies in traffic scenes are formulated under a probabilistic discriminative framework, where contextual constraints during the detection process can be adaptively adjusted in terms of data-dependent neighborhood interaction. Computationally efficient algorithm has been developed to discriminate moving cast shadows and handle nonstationary background processes for real-time vehicle detection in video streams. Experimental results show that the proposed approach effectively fuses contextual dependencies and robustly detects moving vehicles under heavy shadows even in grayscale video. </para>

Asymmetry in the perception of motion in depth induced by moving cast shadows

An expanding object, which may represent an approaching motion, is easier to detect than a contracting one, which may represent a receding object. To confirm the generality of asymmetry in the detection of approaching and receding motions, we focused on the perception of apparent motion in depth created by moving cast shadows. The visual search for an approaching target among receding distractors was more efficient than for the opposite condition (Experiment 1). However, this asymmetry disappeared when a light shadow was added (Experiments 2 and 3). This suggests that the visual system is specialized to detect approaching motion defined by cast shadows, as well as other three-dimensional cues such as expanding motion and shading.

A method to segment moving vehicle cast shadow based on wavelet transform

In many image analysis and interpretation applications, shadows interfere with fundamental tasks such as object extraction and description. According to illumination, shadows interfere with moving vehicle extraction and location and recognition. For this reason, shadow segmentation is an important step in real-time vehicle recognition system. In this paper, we propose a simple and effective method for detection of moving cast shadows on a traffic surveillance scene. The proposed method exploits spectral and geometrical properties of shadows and relationship between the point in shadow region and space position and vehicle shape. Firstly, the cast shadows can be rough detected by spectral properties, and then feature points of occluding function are detected using wave transform, finally, the boundary between self-shadow and cast shadow is detected. The proposed method does not know in advance the light source direction and the color information of vehicle and background texture information. Our experimental results demonstrate that the proposed cast shadows segmentation method can detect the shadows regions accurately and completely. This is the foundation for future vehicle recognition and understanding.

Indoor and outdoor people detection and shadow suppression by exploiting HSV color information

An adaptive background model based on maximum statistical probability and a shadow suppression scheme for indoor and outdoor people detection by exploiting hue saturation value (HSV) color information is proposed. To obtain the initial background scene, the frequency of R, G, and B component values for each pixel at the same position in the learning sequence are respectively calculated; the R, G, and B component values with the biggest ratios are incorporated to model the initial background. The background maintenance, or the socalled background re-initiation, is also proposed to adapt to scene changes such as illumination changes and scene geometry changes. Moving cast shadows generally exhibit a challenge for accurate moving target detection. Based on the observation that a shadow cast on a background region lowers its brightness but does not change its chromaticity significantly, we address this problem in the article by exploiting HSV color information. In addition, quantitative metrics is introduced to evaluate the algorithm on a benchmark suite of indoor and outdoor video sequences. The experimental results are given to show the performance of the algorithm.