Natural Outdoor Scenes Research Articles

Navigating in Virtual Worlds Using a Self-Paced SSVEP-Based Brain–Computer Interface with Integrated Stimulation and Real-Time Feedback

An open question in research nowadays is the usability of brain–computer interfaces (BCI) conceived to extend human capabilities of interaction within a virtual environment. Several paradigms are used for BCI, but the steady-state visual-evoked potential (SSVEP) stands out as it provides a higher information transfer rate while requiring less training. It is an electroencephalographic response detectable when the user looks at a flickering visual stimulus. This research proposes a novel approach for SSVEP-based BCI controller used here for navigation within a 3D virtual environment. For the first time, the flickering stimuli were integrated into virtual objects as a part of the virtual scene in a more transparent and ecological way. As an example, when navigating inside a virtual natural outdoor scene, we could embed the SSVEP flashes in the wings of virtual butterflies surrounding the user. We could also introduce the use of animated and moving stimulations when using SSVEP-based BCI, as the virtual butterflies were left with the possibility of moving and flying in front of the user. Moreover, users received real-time feedback of their mental activity and were thus aware of their detected SSVEP directly and continuously. An experiment has been conducted to assess the influence of both the feedback and the integrated controller on navigation performance and subjective preference. We found that the usage of a controller integrated within the virtual scene along with the feedback seems to improve subjective preference and feeling of presence, despite reduced performance in terms of speed. This suggests that SSVEP-based BCI interfaces for virtual environments could move on from static targets and use integrated and animated stimuli presented in an ecological way for controls in systems where performance demands could be relaxed to benefit an improvement in interaction naturalness.

Outdoor Scene Image Segmentation Based on Background Recognition and Perceptual Organization

In this paper, we propose a novel outdoor scene image segmentation algorithm based on background recognition and perceptual organization. We recognize the background objects such as the sky, the ground, and vegetation based on the color and texture information. For the structurally challenging objects, which usually consist of multiple constituent parts, we developed a perceptual organization model that can capture the nonaccidental structural relationships among the constituent parts of the structured objects and, hence, group them together accordingly without depending on a priori knowledge of the specific objects. Our experimental results show that our proposed method outperformed two state-of-the-art image segmentation approaches on two challenging outdoor databases (Gould data set and Berkeley segmentation data set) and achieved accurate segmentation quality on various outdoor natural scene environments.

A Novel Efficient Algorithm for Locating and Tracking Object Parts in Low Resolution Videos

In this paper, a novel efficient algorithm is presented for locating and tracking object parts in low resolution videos using Lowe's SIFT keypoints with a nearest neighbor object detection approach. Our interest lies in using this information as one step in the process of automatically programming service, household, or personal robots to perform the skills that are being taught in easily obtainable instructional videos. In the reported experiments, the system looked for 14 parts of inanimate and animate objects in 40 natural outdoor scenes. The scenes were frames from a low-resolution instructional video on cleaning golf clubs containing 2,405 frames of 180 by 240 pixels. The system was trained using 39 frames that were half-way between the test frames. Despite the low resolution quality of the instructional video and occluded training samples, the system achieved a recall of 49% with a precision of 71% and an F1 of 0.58, which is better than that achieved by less demanding applications. In order to verify that the reported results were not dependent on the specific video, the proposed technique was applied to another video and the results are reported.

Visual discomfort and natural image statistics.

Images with excessive energy at medium spatial frequencies (Fernandez and Wilkins, 2008 Perception 37 1098-1113), or that have high color contrast and little or no luminance contrast (Wilkins et al, 2008 Perception 37 Supplement, 144-145) appear uncomfortable or aversive and can induce headaches in hypersensitive observers. Such stimuli are uncharacteristic of natural images, and we examined whether visual discomfort more generally increases with deviations from the spatial and chromatic properties of natural scenes. Observers rated the level of discomfort or artistic merit in color images generated from noise or random overlapping rectangles (Mondrians). In one set, the slopes of the amplitude spectra for luminance or chromatic contrast were varied independently to create images ranging from strongly blurred to sharpened relative to a 'natural' 1/f spectrum. Perceived blur was dominated by the luminance slopes, with discomfort rated lowest for approximately 1/f spectra. In a second set of focused Mondrians, color was varied along different axes in the L-M versus S-LM chromatic plane. Discomfort ratings were lowest around bluish yellowish axes that are again typical of natural outdoor scenes. A final set varied in the levels of both luminance and chromatic contrast. Discomfort increased with increasing color contrast or decreasing luminance contrast, and tended toward lower ratings for images with balanced levels of luminance and color contrast. Notably, these ratings of discomfort were not related to judgments of artistic merit. Thus, for both spatial and chromatic content, the least aversive images corresponded to characteristic properties of the natural visual environment, and may reflect a normalization of visual coding to the natural world.

Large-scale conditional random field for natural outdoor scene labeling

We propose a novel large-scale conditional random field model with respect to the problem of natural outdoor scene labeling. The novelty of the proposed method lies in three aspects: 1. features from two neighboring regions are concatenated to form the input of the pair-wise classifier to compensate for the simultaneous feature deviation of neighboring regions; 2. the definition of a generalized neighboring system and the incorporation of direction-specific patterns in conditional random field models based on the generalized neighboring system to better simulate the visual cognition of human being; and 3. the definition of a similarity criterion based on the bags-of-words expression to facilitate the incorporation of semantic patterns. The proposed model is first evaluated over the Corel dataset. Both qualitative and quantitative results show that our model is capable of modeling large-scale spatial relationships between objects in natural outdoor scenes, and achieves better results than other existing conditional random field models. Furthermore, our model is also evaluated over several other natural datasets, which are taken from logged field tests, to further demonstrate the adaptability of our model to different lighting conditions.

Segmentation of non-natural objects in landscape images using ridgelet transform

This study reports about the detection of non-natural structures in outdoor natural scenes. In particular, we present a new approach based on ridgelet transform for the segmentation of man-made objects in landscape scenes. Multiscale directional moments of ridgelet coefficients are used as features along with a principal component analysis (PCA) followed by a linear discriminant analysis (LDA), kernel-based LDA (KLDA), or support vector classifier (SVC). The statistical learning is done on about 3,000 image patches that represent natural and artificial content. Performances are measured in terms of image patch type classification (natural versus non-natural) and man-made object segmentation on two different image test sets. Results using ridgelets are compared to Gabor features. Altogether, we compare performance for six different feature/classifier combinations: ridgelets+LDA, ridgelet+KLDA, ridgelets+SVC, Gabor+LDA, Gabor+KLDA, and Gabor+SVC, and various external parameter values. Results show that most of the time, the combinations with ridgelets provide comparable or better performance.

Variations in normal color vision. V. Simulations of adaptation to natural color environments.

Modern accounts of color appearance differ in whether they assume that the perceptual primaries (e.g., white and the unique hues of red, green, blue, and yellow) correspond to unique states determined by the spectral sensitivities of the observer or by the spectral statistics of the environment. We examined the interaction between observers and their environments by asking how color perception should vary if appearance depends on fixed responses in a set of color channels, when the sensitivities of these channels are adapted in plausible ways to different environments. Adaptation was modeled as gain changes in the cones and in multiple postreceptoral channels tuned to different directions in color-luminance space. Gains were adjusted so that the average channel responses were equated across two environments or for the same environment during different seasons, based on sets of natural outdoor scenes (Webster et al., 2007). Because of adaptation, even observers with a shared underlying physiology should perceive color in significantly and systematically different ways when they are exposed to and thus adapted by different contexts. These include differences in achromatic settings (owing to variations in the average chromaticity of locations) and differences in perceived hue (because of differences in scene contrasts). Modeling these changes provides a way of simulating how colors might be experienced by individuals in different color environments and provides a measure of how much color appearance might be modulated for a given observer by variations in the environment.

Optimal color space for relative color polygons

We aim for detection and recognition of planar objects in natural outdoor scenes under varying illumination conditions. To achieve this, we propose Relative Color Polygons (RCPs) using component colors of objects for color matching. They can be defined on many color spaces, and it is found that a 2D color space (XY space) is the optimal color space for our relative color method compared with other color spaces. To evaluate the invariance to illumination changes for object recognition, experiments have been carried out using 500 outdoor scene images. By using the proposed model, the color matching rate of the input images with the standard one was 95%. This framework is potentially applicable to image retrieval, image segmentation, image recognition, and so on.

Seasonal variations in the color statistics of natural images

We examined how the distribution of colors in natural images varies as the seasons change. Images of natural outdoor scenes were acquired at locations in the Western Ghats, India, during monsoon and winter seasons and in the Sierra Nevada, USA, from spring to fall. The images were recorded with an RGB digital camera calibrated to yield estimates of the L, M, and S cone excitations and chromatic and luminance contrasts at each pixel. These were compared across time and location and were analyzed separately for regions of earth and sky. Seasonal climate changes alter both the average color in scenes and how the colors are distributed around the average. Arid periods are marked by a mean shift toward the +L pole of the L vs. M chromatic axis and a rotation in the color distributions away from the S vs. LM chromatic axis and toward an axis of bluish–yellowish variation, both primarily due to changes in vegetation. The form of the change was similar at the two locations suggesting that the color statistics of natural images undergo a characteristic pattern of temporal variation. We consider the implications of these changes for models of both visual sensitivity and color appearance.

Segmentation and description of natural outdoor scenes

A scene description and segmentation system capable of recognising natural objects (e.g. sky, trees, grass) under different outdoor conditions is presented. We propose an hybrid and probabilistic classifier of image regions as a first step in solving the problem of scene context generation. We focus our work in the problem of image regions labeling to classify every pixel of a given image into one of several predefined classes. The result is both a segmentation of the image and a recognition of each segment as a given object class or as an unknown segmented object. Classification performance has been evaluated with the Outex dataset and compared to the approach of Martí et al. (IVC 2001) and He et al. (CVPR 2004) using their own datasets, showing the superiority of our method.

Mapping multiple visual areas in the human brain with a short fMRI sequence

It is a fundamental insight of neuroscience that the cerebral cortex is divided into spatially separated and functionally distinct areas. In this study, we tried to map a large number of visual areas in individual subjects passively viewing a simple stimulus sequence during functional magnetic resonance imaging (fMRI) at 1.5 T. The blocked stimulus sequence contrasted static object photographs with video takes of movement through natural indoor and outdoor scenes, alternated with a control fixation task. Two runs of the 5-min sequence sufficed to invoke 29 distinguishable activations, 16 (13 bilateral) of which were observed in all 10 participants. At the ventral side, object responsive activations were organized along the lateral occipital–temporal surface and near the collateral and occipital–temporal sulci. The latter activations, corresponding to the lateral occipital complex, showed a different activation profile from those near the collateral sulcus, most likely corresponding to the color constancy areas V4/V8–V4α. A potentially new fusiform object area was seen in 6 subjects, even more anterior than the parahippocampal place area. At the dorsal side, consistent activations were mainly related to motion stimuli and included the well-known areas V3a, VIPS, POIPS, hV5+, STS and the cingulate sulcus. There was consistent activation in the parietal–occipital sulcus, containing the areas V6a and V6. In addition, all subjects showed activation in the superior–anterior precuneus. Thus, the short stimulus sequence robustly invoked multiple visual areas and can be used to map the organization of the visual system in normal and brain-damaged individuals.

Psychophysical estimates of the number of spectral-reflectance basis functions needed to reproduce natural scenes

Theoretical analyses of spectral reflectances of natural surfaces suggest that their perceived colors can be well reproduced by approximations comprising combinations of three or four spectral basis functions. The aim of the present work was to assess psychophysically the number of basis functions necessary to reproduce entire natural outdoor scenes. Hyperspectral images of 20 such scenes were each subjected to a principal component analysis and then reproduced with a variable number of basis functions. The quality of the color approximation under daylight illumination was quantified theoretically in CIELAB space and psychophysically by spatial and temporal two-alternative forced-choice measurements in which the original and the approximated images were compared on a calibrated color monitor. Although five basis functions produced on average unit error in CIELAB space, original images were visually indistinguishable from their approximations only if there were at least eight basis functions. The combination of the spectral diversity of the natural world and the observed levels of color discrimination suggest that estimates of the minimum number of basis functions necessary to reproduce natural scenes may need to be revised upward.

RECOGNITION OF PARTIALLY OCCLUDED PLANT LEAVES USING A MODIFIED WATERSHED ALGORITHM

Occlusion is an obstacle to two-dimensional machine vision recognition of plants in natural outdoor scenes. Five modifications to the Watershed algorithm were investigated for separating occluded plant leaves, in an attempt to reduce the excessive object fragmentation associated with the original Watershed algorithm. The best modified algorithm improved the recognition of occluded tomato cotyledons and tomato true leaves improved by a factor of 3 and 2, respectively, after it was applied to occluded plant leaves in natural outdoor scenes. Two of the modified Watershed algorithms required about 11% less computation time than the original.

Path detection in video surveillance

This paper addresses the problem of automatically extracting frequently used pedestrian pathways from video sequences of natural outdoor scenes. Path models are learnt from the accumulation of trajectory data over long time periods, and can be used to augment the classification of subsequent track data. In particular, labelled paths provide an efficient means for compressing the trajectory data for logging purposes. In addition, the model can be used to compute a probabilistic prediction of the pedestrian's location many time steps ahead, and to aid the recognition of unusual behaviour identified as atypical object motion.

Distance Estimation in a 3-D Imaging Display

Imaging displays have lacked several characteristics of direct vision such as true stereopsis, and this may in part account for some loss in visual performance when compared with direct vision. One such decrement in performance is in judgments of distance. An experiment is reported in which judgments of distance of targets were obtained from videotaped images of targets in a natural outdoor scene. Presentations were made under 3 conditions: stereo with near or far zero disparities and binocular nonstereo ("biocular") viewing. Underconstancy was found to be less for stereo presentation, but only when the point of zero disparity was in the plane of presentation of the virtual image. Some implications for the design of cockpit imaging displays are discussed.

Contrast Adaptation and the Spatial Structure of Natural Images

Contrast (pattern-selective) adaptation influences perception by adjusting sensitivity to the prevailing pattern of stimulation. We asked how the state of adaptation might depend on the patterns of spatial contrast typical of the natural visual environment. In one set of experiments, we examined whether adaptation to the characteristic amplitude spectra of natural images (which tend to decrease with frequency as 1/f) induces characteristic changes in contrast sensitivity. Contrast thresholds and suprathreshold contrast matches were measured after adaptation to random samples from an ensemble of images of natural outdoor scenes, or synthetic images formed by filtering the amplitude spectra of noise over a range of slopes. Adaptation differentially reduced sensitivity at low to medium spatial frequencies, but losses were not strongly dependent on the slope of the adapting spectra. In a second set of experiments, we examined the figural aftereffects induced by adaptation to naturalistic stimuli, by adapting and testing with images of human faces, for which small configural changes are highly discriminable. Observers adapted to frontal-view images of faces that were distorted by local expansions or contractions about the centre, and then adjusted distortions in test images to try to select the original face. Adaptation strongly biased perception in a direction opposite to the adapting distortion, with strongest aftereffects when test and adapting stimuli were derived from the same face image. Our results suggest that adaptation to the stimuli encountered in the course of normal viewing may exert ubiquitous and selective influences that are important in characterising the normal operating state of the visual system.

An integrated approach for scene understanding based on Markov Random Field model

In this paper, we propose a Markov Random Field model-based approach as a unified and systematic way for modeling, encoding and applying scene knowledge to the image understanding problem. In our proposed scheme we formulate the image segmentation and interpretation problem as an integrated scheme and solve it through a general optimization algorithm. More specifically, the image is first segmented into a set of disjoint regions by a conventional region-based segmentation technique which operates on image pixels, and a Region Adjacency Graph (RAG) is then constructed from the resulting segmented regions based on the spatial adjacencies between regions. Our scheme then proceeds on the RAG by defining the region merging and labeling problem based on the MRF models. In the MRF model we specify the a priori knowledge about the optimal segmentation and interpretation in the form of clique functions and those clique functions are incorporated into the energy function to be minimized by a general optimization technique. In the proposed scheme, the image segmentation and interpretation processes cooperate in the simultaneous optimization process such that the erroneous segmentation and misinterpretation due to incomplete knowledge about each problem domain can be compensately recovered by continuous estimation of the single unified energy function. We exploit the proposed scheme to segment and interpret natural outdoor scene images.

Change Detection in Natural Scenes Using Laser Range Measurements from a Mobile Robot

The problem studied in this paper is to detect changes in both indoor scenes and natural outdoor scenes. The sensor is a scanning range measuring laser mounted on a mobile robot. The system should have the capability to detect when objects have been moved, have disappeared or when new objects have entered the area to be monitored. Also, the mobile robot must be able to navigate despite there are changes in the scene.The paper describes a method for simultaneous change detection and pose estimation. The algorithms are presently tested for outdoor scenes. The key tool is the distance transform

Maximum likelihood unsupervised textured image segmentation

This paper presents an algorithm for segmenting an image that is composed of an unknown number of regions c. In each region n, the image data g n are viewed as a realization from a homogeneous parametric random field with a class conditional density function p(g n | γ n ), where γ n is an unknown parameter set. The number of regions c and the segmentation S c are treated as unknown constants that are estimated using the maximum likelihood (ML) estimation principle. The ML estimates for c and S c are obtained by maximizing log{ p(g|S c)} over all possible c and S c . p(g|S c) has the desirable property of unbiasedness; i.e., E S ctrue {log{ p(g|S c )}} ≤ E S ctrue {log{ p(g|S ctrue )}. Unfortunately, it suffers from two limitations: (i) a closed-form analytic expression for p(g|S c ) for a given fixed c cannot be obtained in general, and (ii) in order to arrive at the optimum ( c ∗, S c ∗ ) we must evaluate p(g|S c) for all possible c and S c , a most formidable task. This paper presents a solution to both problems that results into an optimum number of classes c ∗ ; an “optimum” window-based coarse segmentation S c ∗ of the image; and a ML estimate of the parameters Γ S ∗c ∗ = (γ 1, γ 2, …, γ c ∗ ) of the c ∗ regions induced by S c ∗ . From this knowledge, the mixed windows (windows that fall between regions) are segmented further in a supervised mode (known parameter case) using the ML high-resolution segmentation developed by Cohen and Cooper ( IEEE Trans. Pattern Anal. Mach. Intelligence Mar., 1987). The ML algorithm is applied to the problem of unsupervised segmentation of textured images of natural outdoor scenes.

Distance perception as a function of photographic area of view.

Distance perception of depicted objects was examined as a function of photographic area of view. Subjects viewed slides of natural outdoor scenes and directly estimated the distances to specified objects. Area of view was manipulated by means of photographing each scene with lenses of five different focal lengths: 135, 80, 48, 28, and 17 mm. Distance perception along the pictorial depth plane was systematically transformed through changing the photographic area of view: the shorter the focal length of the camera lens, the wider the area of view, and the greater the perceived distance. Linear functions for each subject's distance judgments revealed a very high goodness of fit. Both the y-intercepts and the slopes increased as focal length decreased. The increasing y-intercepts suggest that viewers place themselves farther away from the depicted scene as focal length decreases, compensating for the advancing proximal edge. The increasing slopes suggest that distance information throughout the pictorial depth plane appears to change with focal length. The subjects also made direct judgments of foreground truncation, revealing that foreground truncation decreased as focal length decreased, but that this decrease did not account for the considerable expansion in distance perception.