Information In Natural Scenes Research Articles

Discrimination of natural scenes in central and peripheral vision

We conducted suprathreshold discrimination experiments to compare how natural-scene information is processed in central and peripheral vision (16° eccentricity). Observers’ ratings of the perceived magnitude of changes in naturalistic scenes were lower for peripheral than for foveal viewing, and peripheral orientation changes were rated less than peripheral colour changes. A V1-based Visual Difference Predictor model of the magnitudes of perceived foveal change was adapted to match the sinusoidal grating sensitivities of peripheral vision, but it could not explain why the ratings for changes in peripheral stimuli were so reduced. Perceived magnitude ratings for peripheral stimuli were further reduced by simultaneous presentation of flanking patches of naturalistic images, a phenomenon that could not be replicated foveally, even after M-scaling the foveal stimuli to reduce their size and the distances from the flankers. The effects of the peripheral flankers are very reminiscent of crowding phenomena demonstrated with letters or Gabor patches.

Time course of the integration of spatial frequency-based information in natural scenes

It is known that visual information is processed separately and based on multiple spatial frequencies. Therefore, integration of information is important for categorization of natural scenes. To clarify the time course of visual integration, we examined categorization accuracies for spatially filtered images as a function of image exposure duration. Results indicated that, with image durations of 100-ms, accuracy was superior with spatially integrable images when compared with accuracy levels based upon the probability summation model estimated from accuracies of separately presented low- and high-frequency images. This finding suggests that spatial frequency integration begins earlier than 100-ms after the image onset.

Cones perform a non-linear transformation on natural stimuli

Visual information in natural scenes is distributed over a broad range of intensities and contrasts. This distribution has to be compressed in the retina to match the dynamic range of retinal neurons. In this study we examined how cones perform this compression and investigated which physiological processes contribute to this operation. M- and L-cones of the goldfish were stimulated with a natural time series of intensities (NTSI) and their responses were recorded. The NTSI displays an intensity distribution which is skewed towards the lower intensities and has a long tail into the high intensity region. Cones transform this skewed distribution into a more symmetrical one. The voltage responses of the goldfish cones were compared to those of a linear filter and a non-linear biophysical model of the photoreceptor. The results show that the linear filter under-represents contrasts at low intensities compared to the actual cone whereas the non-linear biophysical model performs well over the whole intensity range used. Quantitative analysis of the two approaches indicates that the non-linear biophysical model can capture 91 +/- 5% of the coherence rate (a biased measure of information rate) of the actual cone, where the linear filter only reaches 48 +/- 8%. These results demonstrate that cone photoreceptors transform an NTSI in a non-linear fashion. The comparison between current clamp and voltage clamp recordings and analysis of the behaviour of the biophysical model indicates that both the calcium feedback loop in the outer segment and the hydrolysis of cGMP are the major components that introduce the specific non-linear response properties found in the goldfish cones.

Neural mechanisms of rapid natural scene categorization in human visual cortex

The visual system has an extraordinary capability to extract categorical information from complex natural scenes. For example, subjects are able to rapidly detect the presence of object categories such as animals or vehicles in novel scenes that are presented very briefly1,2. This is even true when subjects do not pay attention to the scenes and simultaneously perform an unrelated attentionally demanding task3, a stark contrast to the capacity limitations predicted by most theories of visual attention4,5. Here, we use functional magnetic resonance imaging and an object categorization task, in which subjects detected the presence of people or cars in briefly presented natural scenes, to provide a neural basis for rapid natural scene categorization in visual cortex. The pattern of neural activity in object-selective cortex evoked by the natural scenes contained information regarding the target category, even in scenes that were task-irrelevant and presented outside the focus of spatial attention. These findings suggest that the rapid detection of categorical information in natural scenes is mediated through a category-specific biasing mechanism in object-selective cortex that operates in parallel across the visual field and biases information processing in favour of objects belonging to the target object category.

Boundary Cue Invariance in Cortical Orientation Maps

We effortlessly perceive oriented boundaries defined by either luminance changes ('first-order' cues) or texture variations ('second-order' cues). Many neurons in mammalian visual cortex show orientation preference to both types of boundaries, but it is uncertain how they contribute to perceptual orientation cue-invariance at the neuronal population level. Using optical imaging in cat A 18, we observed highly similar orientation preference maps to first-order and a variety of second-order visual stimuli. Thus the neuronal representation of coarse-scale boundary orientation appears to be invariant to the characteristics (including local orientation) of the fine-scale textures by which those boundaries are defined. A common feature of second-order visual stimuli is that modulation shifts their Fourier energy for boundary orientation to the higher spatial frequencies of their constituent textures - our results suggest a common neural mechanism (demodulation) mediating visual processing of many kinds of texture boundary. The similarity between orientation maps to different stimuli implies that second-order responsive neurons are homogeneously distributed across the cortical surface. Such homogeneously cue-invariant orientation representation could provide a neural substrate for perceptual form-cue invariance, and reflect an optimal organization for encoding orientation information in natural scenes.

Constructing visual representations of natural scenes: the roles of short- and long-term visual memory.

A "follow-the-dot" method was used to investigate the visual memory systems supporting accumulation of object information in natural scenes. Participants fixated a series of objects in each scene, following a dot cue from object to object. Memory for the visual form of a target object was then tested. Object memory was consistently superior for the two most recently fixated objects, a recency advantage indicating a visual short-term memory component to scene representation. In addition, objects examined earlier were remembered at rates well above chance, with no evidence of further forgetting when 10 objects intervened between target examination and test and only modest forgetting with 402 intervening objects. This robust prerecency performance indicates a visual long-term memory component to scene representation.

Chromatic structure of natural scenes

We applied independent component analysis (ICA) to hyperspectral images in order to learn an efficient representation of color in natural scenes. In the spectra of single pixels, the algorithm found basis functions that had broadband spectra and basis functions that were similar to natural reflectance spectra. When applied to small image patches, the algorithm found some basis functions that were achromatic and others with overall chromatic variation along lines in color space, indicating color opponency. The directions of opponency were not strictly orthogonal. Comparison with principal-component analysis on the basis of statistical measures such as average mutual information, kurtosis, and entropy, shows that the ICA transformation results in much sparser coefficients and gives higher coding efficiency. Our findings suggest that nonorthogonal opponent encoding of photoreceptor signals leads to higher coding efficiency and that ICA may be used to reveal the underlying statistical properties of color information in natural scenes.

Color and luminance information in natural scenes: errata

Get PDF Email Share Share with Facebook Tweet This Post on reddit Share with LinkedIn Add to CiteULike Add to Mendeley Add to BibSonomy Get Citation Copy Citation Text C. A. Párraga, G. Brelstaff, T. Troscianko, and I. R. Moorhead, "Color and luminance information in natural scenes: errata," J. Opt. Soc. Am. A 15, 1708-1708 (1998) Export Citation BibTex Endnote (RIS) HTML Plain Text Citation alert Save article

Color and luminance information in natural scenes.

The spatial filtering applied by the human visual system appears to be low pass for chromatic stimuli and band pass for luminance stimuli. Here we explore whether this observed difference in contrast sensitivity reflects a real difference in the components of chrominance and luminance in natural scenes. For this purpose a digital set of 29 hyperspectral images of natural scenes was acquired and its spatial frequency content analyzed in terms of chrominance and luminance defined according to existing models of the human cone responses and visual signal processing. The statistical 1/f amplitude spatial-frequency distribution is confirmed for a variety of chromatic conditions across the visible spectrum. Our analysis suggests that natural scenes are relatively rich in high-spatial-frequency chrominance information that does not appear to be transmitted by the human visual system. This result is unlikely to have arisen from errors in the original measurements. Several reasons may combine to explain a failure to transmit high-spatial-frequency chrominance: (a) its minor importance for primate visual tasks, (b) its removal by filtering applied to compensate for chromatic aberration of the eye's optics, and (c) a biological bottleneck blocking its transmission. In addition, we graphically compare the ratios of luminance to chrominance measured by our hyperspectral camera and those measured psychophysically over an equivalent spatial-frequency range.

Natural-Image Discrimination in the Periphery: The Importance of Phase and Amplitude Information

Previous studies with simplified stimuli such as combinations of sinusoidal gratings have revealed phase identification losses in the periphery that are not eliminated by a scaling factor. How do these phase processing problems influence our ability to discriminate natural images in the periphery? In this study the ability of an observer to identify the ‘odd-image-out’ when there is either an amplitude-only, phase-only, or amplitude and phase change in one out of three stimuli is compared. Pairs of Fourier-manipulated black-and-white digitised photographs of natural images were used and phase and amplitude spectral exchanges of varying proportions were made between two different images. Measurements were made to determine the smallest phase change needed in order for the observer to reliably discriminate the manipulated image, compared to two reference stimuli, at eccentricities of 0°, 2.5°, 5°, and 10°. This was compared to discrimination thresholds found when amplitude and phase, and amplitude alone were exchanged. The ability to discriminate images on the basis of phase information alone did fall off quickly with eccentricity (comparable to phase and amplitude discriminations). However, there was a much more rapid decline in amplitude-only discrimination. It appears that phase information in natural scenes remains a relatively more important visual cue in the periphery than amplitude.

Color Vision in Robotic Fruit Harvesting

ABSTRACT THIS report discusses the use of chrominance information in natural scenes to enhance digital color images used to control a robotic manipulator for fruit harvest. By exploiting the natural high contrast in color between fruit and other objects in a color image the ability to differentiate citrus fruit from background leaves, soil, and sky is demonstrated.