Scene Variations Research Articles

Complex background subtraction by pursuing dynamic spatio-temporal models.

Although it has been widely discussed in video surveillance, background subtraction is still an open problem in the context of complex scenarios, e.g., dynamic backgrounds, illumination variations, and indistinct foreground objects. To address these challenges, we propose an effective background subtraction method by learning and maintaining an array of dynamic texture models within the spatio-temporal representations. At any location of the scene, we extract a sequence of regular video bricks, i.e., video volumes spanning over both spatial and temporal domain. The background modeling is thus posed as pursuing subspaces within the video bricks while adapting the scene variations. For each sequence of video bricks, we pursue the subspace by employing the auto regressive moving average model that jointly characterizes the appearance consistency and temporal coherence of the observations. During online processing, we incrementally update the subspaces to cope with disturbances from foreground objects and scene changes. In the experiments, we validate the proposed method in several complex scenarios, and show superior performances over other state-of-the-art approaches of background subtraction. The empirical studies of parameter setting and component analysis are presented as well.

Companied Edge Flow Histogram and Color Histogram to Represent Tracking Objects

Tracking moving objects from moving platform in videos sequence is a challenging task .object movement and platform movement are sources of variations in scene. Mean Shift Algorithm (MSA) is the common tracking algorithm due simple and efficient procedure. Correct Background Weight Histogram(CBWH)decrease background effect in target representation module . The main drawbacks in MSA is the ineffective model representation to handle illumination variation and occultation problems . MSA failed to track objects in video containing wide ranges of variations and background motion. In this paper motion information is exploited from edge flow by gradient differential. Histogram for edge flow combined with color histogram called Motion Flow Histograms (MFH).MFH used to represent tracking target between two successive frames. New module target representation reduces the false tracker rate without evident increasing time compare to the classical tracking MSA and CBWH. General Terms Moving platform, tracking algorithm

Motion Objects Segmentation and Shadow Suppressing without Background Learning

An approach to segmenting motion objects and suppressing shadows without background learning has been developed. Since wavelet transformation indicates the position of sharper variation, it is adopted to extract the information contents with the most meaningful features based on two successive video frames only. According to the fact that the saturation component is lower in the region of shadow and is independent of the brightness, HSV color space is selected to extract foreground motion region and suppress shadow instead of other color models. A local adaptive thresholding approach is proposed to extract initial binary motion masks based on the results of the wavelet transformation. A foreground reclassification is developed to get an optimal segmentation by fusion of mode filtering, connectivity analysis, and spatial-temporal correlation. Comparative studies with some investigated methods have indicated the superior performance of the proposal in extracting motion objects and suppressing shadows from cluttered contents with dynamic scene variation and crowded environments.

Experience-based navigation for long-term localisation

This paper is about long-term navigation in environments whose appearance changes over time, suddenly or gradually. We describe, implement and validate an approach which allows us to incrementally learn a model whose complexity varies naturally in accordance with variation of scene appearance. It allows us to leverage the state of the art in pose estimation to build over many runs, a world model of sufficient richness to allow simple localisation despite a large variation in conditions. As our robot repeatedly traverses its workspace, it accumulates distinct visual experiences that in concert, implicitly represent the scene variation: each experience captures a visual mode. When operating in a previously visited area, we continually try to localise in these previous experiences while simultaneously running an independent vision-based pose estimation system. Failure to localise in a sufficient number of prior experiences indicates an insufficient model of the workspace and instigates the laying down of the live image sequence as a new distinct experience. In this way, over time we can capture the typical time-varying appearance of an environment and the number of experiences required tends to a constant. Although we focus on vision as a primary sensor throughout, the ideas we present here are equally applicable to other sensor modalities. We demonstrate our approach working on a road vehicle operating over a 3-month period at different times of day, in different weather and lighting conditions. We present extensive results analysing different aspects of the system and approach, in total processing over 136,000 frames captured from 37 km of driving.

Background subtraction using hybrid feature coding in the bag-of-features framework

Although numerous algorithms have been proposed for background subtraction with demonstrated success, it remains a challenging problem. One of the main reasons is the lack of effective background model to account for the complex variations of backgrounds. Although researchers have strived to obtain a background model effectively attenuating false positives from dynamic background variations, their methods are still sensitive to structured motion patterns of background (e.g., waving leaves, rippling water, spouting fountain, etc.). In this paper, inspired by the bag-of-features framework, we present a simple, novel, yet powerful approach for background subtraction. It relies on the hypothesis that texture variations in the background scenes can be well attenuated by effectively encoding the local color and texture information. Specifically, the proposed method adopts joint domain-range features, which are encoded in the soft-assignment coding procedure. We also propose a novel method for deciding the appropriate kernel variances in the soft-assignment coding, which result in strong adaptability and robustness to dynamic scenes compared to employing fixed kernel variances. Experimental results demonstrate that our proposed method is able to handle severe textural variations of backgrounds and perform favorably against the state-of-the-art methods.

Coastal Flood Inundation Monitoring with Satellite C‐band and L‐band Synthetic Aperture Radar Data

AbstractSatellite Synthetic Aperture Radar (SAR) was evaluated as a method to operationally monitor the occurrence and distribution of storm‐ and tidal‐related flooding of spatially extensive coastal marshes within the north‐central Gulf of Mexico. Maps representing the occurrence of marsh surface inundation were created from available Advanced Land Observation Satellite (ALOS) Phased Array type L‐Band SAR (PALSAR) (L‐band) (21 scenes with HH polarizations in Wide Beam [100 m]) data and Environmental Satellite (ENVISAT) Advanced SAR (ASAR) (C‐band) data (24 scenes with VV and HH polarizations in Wide Swath [150 m]) during 2006‐2009 covering 500 km of the Louisiana coastal zone. Mapping was primarily based on a decrease in backscatter between reference and target scenes, and as an extension of previous studies, the flood inundation mapping performance was assessed by the degree of correspondence between inundation mapping and inland water levels. Both PALSAR‐ and ASAR‐based mapping at times were based on suboptimal reference scenes; however, ASAR performance seemed more sensitive to reference‐scene quality and other types of scene variability. Related to water depth, PALSAR and ASAR mapping accuracies tended to be lower when water depths were shallow and increased as water levels decreased below or increased above the ground surface, but this pattern was more pronounced with ASAR. Overall, PALSAR‐based inundation accuracies averaged 84% (n = 160), while ASAR‐based mapping accuracies averaged 62% (n = 245).

Region-Based Weighted Prediction for Coding Video With Local Brightness Variations

This paper presents a new region-based scheme for the estimation of weighted prediction (WP) parameter sets for encoders of the H.264/MPEG-4 AVC standard. The proposed scheme is specifically designed for handling local brightness variations (LBVs) in video scenes. It is achieved by making use of multiple WP parameter sets for various regions and assigning them to the same reference frame. An accurate estimation of multiple WP parameter sets is accomplished by: 1) partitioning regions with a simple WP parameter estimator; 2) selecting regions where WP should be applied; and 3) estimating accurate WP parameter sets with a quasioptimal WP parameter estimator. The multiple WP parameter sets of different regions are encoded using the framework of multiple reference frames in the H.264/MPEG-4 AVC standard. With this arrangement, the proposed scheme is compliant with the H.264/MPEG-4 AVC standard. To reduce the implementation cost, a reduction of the memory requirement is realized via look-up tables (LUTs). Experimental results show that the region-based scheme can efficiently handle scenes with global and LBVs and achieve significant coding gain over other WP schemes. Furthermore, our scheme with LUTs can reduce the memory requirement by about 80% while keeping the same coding efficiency as that without LUTs.

The Effect of Scene Variation on the Redundant Use of Color in Definite Reference

This study investigates to what extent the amount of variation in a visual scene causes speakers to mention the attribute color in their definite target descriptions, focusing on scenes in which this attribute is not needed for identification of the target. The results of our three experiments show that speakers are more likely to redundantly include a color attribute when the scene variation is high as compared with when this variation is low (even if this leads to overspecified descriptions). We argue that these findings are problematic for existing algorithms that aim to automatically generate psychologically realistic target descriptions, such as the Incremental Algorithm, as these algorithms make use of a fixed preference order per domain and do not take visual scene variation into account.

Large scale continuous visual event recognition using max-margin Hough transformation framework

In this paper we propose a novel method for continuous visual event recognition (CVER) on a large scale video dataset using max-margin Hough transformation framework. Due to high scalability, diverse real environmental state and wide scene variability direct application of action recognition/detection methods such as spatio-temporal interest point (STIP)-local feature based technique, on the whole dataset is practically infeasible. To address this problem, we apply a motion region extraction technique which is based on motion segmentation and region clustering to identify possible candidate “event of interest” as a preprocessing step. On these candidate regions a STIP detector is applied and local motion features are computed. For activity representation we use generalized Hough transform framework where each feature point casts a weighted vote for possible activity class centre. A max-margin frame work is applied to learn the feature codebook weight. For activity detection, peaks in the Hough voting space are taken into account and initial event hypothesis is generated using the spatio-temporal information of the participating STIPs. For event recognition a verification Support Vector Machine is used. An extensive evaluation on benchmark large scale video surveillance dataset (VIRAT) and as well on a small scale benchmark dataset (MSR) shows that the proposed method is applicable on a wide range of continuous visual event recognition applications having extremely challenging conditions.

A Learning Approach Towards Detection and Tracking of Lane Markings

Road scene analysis is a challenging problem that has applications in autonomous navigation of vehicles. An integral component of this system is the robust detection and tracking of lane markings. It is a hard problem primarily due to large appearance variations in lane markings caused by factors such as occlusion (traffic on the road), shadows (from objects like trees), and changing lighting conditions of the scene (transition from day to night). In this paper, we address these issues through a learning-based approach using visual inputs from a camera mounted in front of a vehicle. We propose the following: 1) a pixel-hierarchy feature descriptor to model the contextual information shared by lane markings with the surrounding road region; 2) a robust boosting algorithm to select relevant contextual features for detecting lane markings; and 3) particle filters to track the lane markings, without knowledge of vehicle speed, by assuming the lane markings to be static through the video sequence and then learning the possible road scene variations from the statistics of tracked model parameters. We investigate the effectiveness of our algorithm on challenging daylight and night-time road video sequences.

Robust Face Detection Based on Knowledge-Directed Specification of Bottom-Up Saliency

This paper presents a novel approach to face detection by localizing faces as the goal-specific saliencies in a scene, using the framework of selective visual attention of a human with a particular goal in mind. The proposed approach aims at achieving human-like robustness as well as efficiency in face detection under large scene variations. The key is to establish how the specific knowledge relevant to the goal interacts with the bottom-up process of external visual stimuli for saliency detection. We propose a direct incorporation of the goal-related knowledge into the specification and/or modification of the internal process of a general bottom-up saliency detection framework. More specifically, prior knowledge of the human face, such as its size, skin color, and shape, is directly set to the window size and color signature for computing the center of difference, as well as to modify the importance weight, as a means of transforming into a goal-specific saliency detection. The experimental evaluation shows that the proposed method reaches a detection rate of 93.4% with a false positive rate of 7.1%, indicating the robustness against a wide variation of scale and rotation.

Haar invariant signatures and spatial recognition using omnidirectional visual information only

This paper describes a method for spatial representation, place recognition and qualitative self-localization in dynamic indoor environments, based on omnidirectional images. This is a difficult problem because of the perceptual ambiguity of the acquired images, and their weak robustness to noise, geometrical and photometric variations of real world scenes. The spatial representation is built up invariant signatures using Invariance Theory where we suggest to adapt Haar invariant integrals to the particular geometry and image transformations of catadioptric omnidirectional sensors. It follows that combining simple image features in a process of integration over visual transformations and robot motion, can build discriminant percepts about robot spatial locations. We further analyze the invariance properties of the signatures and the apparent relation between their similarity measures and metric distances. The invariance properties of the signatures can be adapted to infer a hierarchical process, from global room recognition to local and coarse robot localization. The approach is validated in real world experiments and compared to some local and global state-of-the-art methods. The results demonstrate a very interesting performance of the proposed approach and show distinctive behaviors of global and local methods. The invariant signature method, while being very time and memory efficient, provides good separability results similarly to approaches based on local features.

Is color patchy?

In many natural scenes, shadows and shading, which are primarily luminance-defined features, proliferate. Hence one might expect that the chromatic variations of natural scenes, which more faithfully represent the layout of object surfaces, will contain relatively fewer and larger uniform regions than the luminance variations, i.e., will be more “patchy.” This idea was tested using images of natural scenes that were decomposed into chromatic and luminance layers modeled as the responses of the red‐green, blue‐yellow, and luminance channels of the human visual system. Patchiness was defined as the portion of pixels falling within a ± threshold in the bandpass-filtered image, averaged across multiple filter scales. The red‐green layers were found to be the most patchy, followed by the blue‐yellow layers, with the luminance layers the least patchy. The correlation between image-layer patchiness and the slope of the Fourier amplitude spectrum was small and negative for all layers, the maximum value being for red‐green 0.48. We conclude that the chromatic layers of natural scenes contain more uniform areas than the luminance layers and that this is unpredicted by the slope of the Fourier amplitude spectrum. © 2008 Optical Society of America OCIS codes: 330.0330, 330.1720.

Statistical modeling for the detection, localization and extraction of text from heterogeneous textual images using combined feature scheme

Discriminating between the text and non text regions of an image is a complex and challenging task. In contrast to Caption text, Scene text can have any orientation and may be distorted by the perspective projection. Moreover, it is often affected by variations in scene and camera parameters such as illumination, focus, etc. These variations make the design of unified text extraction from various kinds of images extremely difficult. This paper proposes a statistical unified approach for the extraction of text from hybrid textual images (both Scene text and Caption text in an image) and Document images with variations in text by using carefully selected features with the help of multi level feature priority (MLFP) algorithm. The selected features are combinedly found to be the good choice of feature vectors and have the efficacy to discriminate between text and non text regions for Scene text, Caption text and Document images and the proposed system is robust to illumination, transformation/perspective projection, font size and radially changing/angular text. MLFP feature selection algorithm is evaluated with three common ML algorithms: a decision tree inducer (C4.5), a naive Bayes classifier, and an instance based K-nearest neighbour learner and effectiveness of MLFP is shown by comparing with three feature selection methods with benchmark dataset. The proposed text extraction system is compared with the Edge based method, Connected component method and Texture based method and shown encouraging result and finds its major application in preprocessing for optical character recognition technique and multimedia processing, mobile robot navigation, vehicle license detection and recognition, page segmentation and text-based image indexing, etc.

Chromatic diversity index – an approach based on natural scenes

Common descriptors of light quality fail to predict the chromatic diversity produced by the same illuminant in different contexts such as images of natural scenes. The aim of this paper was to introduce a new index, capable of predicting illuminant-induced variations in the chromatic diversity off natural scenes. The spectral reflectance of each pixel of 50 images of natural scenes obtained using a hyperspectral imaging and the spectral reflectance of 1264 Munsell surfaces were converted into the CIELAB color space for each of the 55 illuminants and 5 light sources. The CIELAB volume was estimated by the convex hull method. The number of discernible colors was estimated by segmenting the CIELAB color volume into unitary cubes and by counting the number of non-empty cubes. High correlation was found between the CIELAB volume occupied by the Munsell surfaces, the number of discernible colors and CILEAB color volume of the colors of natural scenes. These results seem to indicate that a new illuminant chromatic diversity index based on natural scenes could be defined using the CIELAB volume of the Munsell surfaces.

Brightness-compensated 3-D optical flow algorithm for monitoring cochlear motion patterns

A method for three-dimensional motion analysis designed for live cell imaging by fluorescence confocal microscopy is described. The approach is based on optical flow computation and takes into account brightness variations in the image scene that are not due to motion, such as photobleaching or fluorescence variations that may reflect changes in cellular physiology. The 3-D optical flow algorithm allowed almost perfect motion estimation on noise-free artificial sequences, and performed with a relative error of <10% on noisy images typical of real experiments. The method was applied to a series of 3-D confocal image stacks from an in vitro preparation of the guinea pig cochlea. The complex motions caused by slow pressure changes in the cochlear compartments were quantified. At the surface of the hearing organ, the largest motion component was the transverse one (normal to the surface), but significant radial and longitudinal displacements were also present. The outer hair cell displayed larger radial motion at their basolateral membrane than at their apical surface. These movements reflect mechanical interactions between different cellular structures, which may be important for communicating sound-evoked vibrations to the sensory cells. A better understanding of these interactions is important for testing realistic models of cochlear mechanics.

High-level aftereffects to global scene properties.

Adaptation is ubiquitous in the human visual system, allowing recalibration to the statistical regularities of its input. Previous work has shown that global scene properties such as openness and mean depth are informative dimensions of natural scene variation useful for human and machine scene categorization (Greene & Oliva, 2009b; Oliva & Torralba, 2001). A visual system that rapidly categorizes scenes using such statistical regularities should be continuously updated, and therefore is prone to adaptation along these dimensions. Using a rapid serial visual presentation paradigm, we show aftereffects to several global scene properties (magnitude 8-21%). In addition, aftereffects were preserved when the test image was presented 10 degrees away from the adapted location, suggesting that the origin of these aftereffects is not solely due to low-level adaptation. We show systematic modulation of observers' basic-level scene categorization performances after adapting to a global property, suggesting a strong representational role of global properties in rapid scene categorization.

The nesting of search contexts within natural scenes: Evidence from contextual cuing.

In a contextual cuing paradigm, we examined how memory for the spatial structure of a natural scene guides visual search. Participants searched through arrays of objects that were embedded within depictions of real-world scenes. If a repeated search array was associated with a single scene during study, then array repetition produced significant contextual cuing. However, expression of that learning was dependent on instantiating the original scene in which the learning occurred: Contextual cuing was disrupted when the repeated array was transferred to a different scene. Such scene-specific learning was not absolute, however. Under conditions of high scene variability, repeated search array were learned independently of the scene background. These data suggest that when a consistent environmental structure is available, spatial representations supporting visual search are organized hierarchically, with memory for functional subregions of an environment nested within a representation of the larger scene.

Specification of efficient block-matching scheme for motion estimation in video compression

We present an adaptive algorithm that finds the best block-matching results in a computationally constrained and varied environment. The conventional diamond search algorithm, though faster than most known algorithms, is not very robust for sequences with scene variations or significant global motion. To solve this issue, rather than only using one fast motion estimation algorithm, we devise a more adaptive selection of fast motion estimation algorithms. Our adaptive selection approach for fast block search (ASFBS) algorithm uses a diamond search and two new subalgorithms: a cross-three-step search algorithm for large moving images and an advanced cross-diamond search algorithm for small moving images. The proposed ASFBS adapts based on the length of the motion vector, the number of search points, and the matching criteria of the neighboring block. Experimental results show that ASFBS is much more robust; it is faster than other popular fast block-matching algorithms, with smaller distortions.

Spectral Fluctuation Method: A Texture-Based Method to Extract Text Regions in General Scene Images

To obtain text information included in a scene image, we first need to extract text regions from the image before recognizing the text. In this paper, we examine human vision and propose a novel method to extract text regions by evaluating textural variation. Human beings are often attracted by textural variation in scenes, which causes foveation. We frame a hypothesis that texts also have similar property that distinguishes them from the natural background. In our method, we calculate spatial variation of texture to obtain the distribution of the degree of likelihood of text region. Here we evaluate the changes in local spatial spectrum as the textural variation. We investigate two options to evaluate the spectrum, that is, those based on one- and two-dimensional Fourier transforms. In particular, in this paper, we put emphasis on the one-dimensional transform, which functions like the Gabor filter. The proposal can be applied to a wide range of characters mainly because it employs neither templates nor heuristics concerning character size, aspect ratio, specific direction, alignment, and so on. We demonstrate that the method effectively extracts text regions contained in various general scene images. We present quantitative evaluation of the method by using databases open to the public.