Local Luminance Research Articles

From filters to features: location, orientation, contrast and blur.

Consider three main ideas about spatial filtering and feature coding in human spatial vision. (1) Computational theory: the representation of local luminance features--bars and edges--is a crucial step in human vision, forming the basis for many decisions in pattern discrimination. (2) Algorithm: features may be located and characterized in terms of polarity, blur and contrast by comparison of 1st, 2nd and 3rd spatial derivatives taken at a common point. Edges in compound (f + 3f) gratings are seen at or close to peaks of gradient magnitude. More tentatively, bars may be located at peaks of the 2nd derivative or at peaks in the Hilbert transform of the 1st derivative. Peaks of contrast energy do not predict all the features seen. An algorithm for recovering the blur of edges is derived as the square-root of the ratio of 1st to 3rd derivatives at the edge location. This successfully predicts blur matching between Gaussian edges and a variety of other test waveforms, including sine waves. Blur matching is (nearly) contrast invariant, as predicted by this ratio rule. (3) IMPLEMENTATION: experiments on the perception and discrimination of plaids imply that the outputs of tuned filters are combined before feature coding. The adaptive, weighted summation of bandpass filters may serve to synthesize the derivative operators while facilitating the segmentation of overlapping features and preventing the representation from being swamped by noise.

Defining the detection mechanisms for symmetric and rectified flicker stimuli

Symmetric flicker modulates about a background light level and effects no change in the time-average luminance. Rectified flicker is achieved by modulating a luminance-increment and results in both a flickering component and an increase in the time-averaged luminance (luminance-pedestal) above the adapting background light level. We studied the effect that changes in adapting light level and local luminance (within the area of the flickering target) have on thresholds. We measured thresholds for single and multiple cycles of flicker over a range of adapting light levels (Threshold versus Intensity paradigm) and defined their gain as a function of luminance-pedestal amplitude (Threshold versus Amplitude paradigm). The dynamics of symmetric and rectified flicker responses were determined using a Stimulus Onset Asynchrony paradigm. The data show rectified flicker thresholds differ from symmetric flicker thresholds due to two factors that can be contrast-dependent or contrast-independent: (1) local adaptation, which varies with stimulus duration and (2) surround interactions that depend on adapting light level. The dynamics of the thresholds for symmetric and rectified flicker stimuli suggest the detection mechanisms occur early in the visual pathways, involving the magnocellular pathway.

A Real-Time Image-Feature-Extraction and Vector-Generation VLSI Employing Arrayed-Shift-Register Architecture

A feature-extraction and vector-generation VLSI has been developed for real-time image recognition. An arrayed-shift-register architecture has been employed in conjunction with a pipelined directional-edge-filtering circuitry. As a result, it has become possible to scan an image, pixel by pixel, with a 64 x 64-pixel recognition window and generate a 64-dimensional feature vector in every 64 clock cycles. In order to determine the threshold for edge-filtering operation adaptive to local luminance variation, a high-speed median circuit has been developed. A binary median search algorithm has been implemented using high-precision majority voting circuits working in the mixed-signal principle. A prototype chip was designed and fabricated in a 0.18-mum 5-metal CMOS technology. A high-speed feature vector generation in less than 9.7 ns/vector element has been experimentally demonstrated. It is possible to scan a VGA-size image at a rate of 6.1 frames/s, thus generating as many as 1.5 x 106 feature vectors per second for recognition. This is more than 103 times faster than software processing running on a 3-GHz general-purpose processor.

Detecting low shape-frequencies in smooth and jagged contours

It is often assumed that a two-stage filter-process is involved in the coding of visual contours. In the first stage the contour is coded by localized luminance filters selective for, among other dimensions, orientation and spatial frequency. A second stage then integrates the outputs of these local luminance filters over space. In the experiments described here we address the issue of which spatial scales of early luminance filters are involved in the detection of a contour’s deviation from linearity (‘co-linearity failure’), especially at low contour frequencies, for both smooth contours and jagged edges. We also address the question whether it is the orientation or position of the first-stage luminance filters that is used by the second stage. We report two main conclusions. Firstly, across a wide range of shape frequencies, we find that detection thresholds are relatively independent of the spatial scale of the luminance information present in the contour, indicating that detection of co-linearity failure can be effectively mediated by luminance filters tuned to a range of spatial scales. Secondly, we find that detection of co-linearity failure in low shape-frequency contours is primarily based on the local positions, not orientations, of the first-stage luminance filters. As our results suggest that the contour’s local orientation may nevertheless play a role, we hypothesize that the local orientation of the contour is not signaled by luminance filters directly but rather by second-order filters acting on the local positions of the contour.

VEPs elicited by local correlations and global symmetry: Characteristics and interactions

Psychophysical and fMRI studies have indicated that visual processing of global symmetry has distinctive scaling properties, and proceeds more slowly than analysis of contrast, spatial frequency, and texture. We therefore undertook a visual evoked potential (VEP) study to directly compare the dynamics of symmetry and texture processing, and to determine the extent to which they interact. Stimuli consisted of interchange between structured and random black-and-white checkerboard stimuli. For symmetry, structured stimuli were colored with 2-fold symmetry (horizontal or vertical mirror), 4-fold symmetry (both mirror axes), and 8-fold symmetry (oblique mirror axes added). For texture, structured stimuli were colored according to the “even” isodipole texture [Julesz, B., Gilbert, E. N., & Victor, J. D. (1978). Visual discrimination of textures with identical third-order statistics. Biological Cybernetics, 31, 137–140]. Thus, all stimuli had the same contrast, and check size, but differed substantially in correlation structure. To separate components of the VEP related to symmetry and texture from components that could be generated by local luminance and contrast changes, we extracted the odd-harmonic components of the VEP (recorded at C z – O z , C z – O 1, C z – O 2, C z – P z ) elicited by structured–random interchange. Responses to symmetry were largest for the 8-fold patterns, and progressively smaller for 4-fold, vertical, and horizontal symmetry patterns. Eightfold patterns were therefore used in the remainder of the study. The symmetry response is shifted to larger checks and lower temporal frequencies compared to the response to texture, and its temporal tuning is broader. Processing of symmetry makes use of neural mechanisms with larger receptive fields, and slower, more sustained temporal tuning characteristics than those involved in the analysis of texture. Sparse stimuli were used to dissociate check size and check density. VEP responses to sparse symmetry stimuli showed that there is no difference between first- and second-order symmetry for densities less than 12.5%. We discuss these findings in relation to local and global visual processes.

Luminance-Modulated Adaptation of Global Flash mfERG: Fellow Eye Losses in Asymmetric Glaucoma

To use the global flash multifocal electroretinogram (mfERG) in patients with asymmetric glaucoma to determine whether retinal function is affected in fellow eyes that have no glaucomatous visual field defects. Forty normal subjects and 12 patients with asymmetric glaucoma were recruited for visual field and mfERG measurement. The mfERG was assessed by using a global-flash stimulation paradigm with four video frames: 103 scaled hexagonal elements followed by a dark frame, a global-flash frame, and a dark frame. The localized luminance difference was set at 96%, 65%, 49%, and 29% display contrast during the four different test conditions, respectively. The first-order kernel response was measured, and the "adaptive index" which has been used previously was calculated. In fellow eyes with normal visual fields, the amplitude of the induced component (IC) was significantly reduced, and the adaptive index was reduced by a factor of almost 10 (P < 0.0001), as it was in the glaucomatous eyes. Although the adaptive index in the better (fellow) eye of the patients with glaucoma was slightly higher than in the eyes with diagnosed glaucoma, these differences were not statistically significant. The significant reduction of the adaptive index in the better eyes in subjects with asymmetric glaucoma shows that the fast adaptive mechanism(s) were reduced in these eyes. This implies that eyes that have functionally normal visual acuity and visual fields have abnormal fast-adaptive mechanisms.

Responses to Lightness Variations in Early Human Visual Cortex

Lightness is the apparent reflectance of a surface, and it depends not only on the actual luminance of the surface but also on the context in which the surface is viewed [1-10]. The cortical mechanisms of lightness processing are largely unknown, and the role of early cortical areas is still a matter of debate [11-17]. We studied the cortical responses to lightness variations in early stages of the human visual system with functional magnetic resonance imaging (fMRI) while observers were performing a demanding fixation task. The set of dynamically presented visual stimuli included the rectangular version of the classic Craik-O'Brien stimulus [3, 18, 19] and a variant that led to a weaker lightness effect, as well as a pattern with actual luminance variations. We found that the cortical activity in retinotopic areas, including the primary visual cortex (V1), is correlated with context-dependent lightness variations.

Responses of Neurons in Primary Visual Cortex to Transient Changes in Local Contrast and Luminance

During normal saccadic inspection of natural images, the receptive fields of cortical neurons are bombarded with frequent simultaneous changes in local mean luminance and contrast, yet there have been no systematic studies of how cortical neurons respond to such stimulation. The responses of single neurons in the primary visual cortex of the cat were measured for 200 ms presentations of sine-wave gratings confined to the conventional receptive field. Both local mean luminance and contrast were parametrically and randomly varied over the 1-1.5 log unit ranges that are typical of natural images. We find that responses are strongly modulated by both the local mean luminance and contrast, but in an approximately separable manner: the contrast response function is approximately invariant except for a scale factor that depends on the local mean luminance. The shape of the temporal response profiles were found to be approximately invariant with contrast, but were strongly affected by the local mean luminance. The results suggest that most, if not all, cortical neurons carry substantial local luminance information.

A Biologically Motivated Multiresolution Approach to Contour Detection

Standard edge detectors react to all local luminance changes, irrespective of whether they are due to the contours of the objects represented in a scene or due to natural textures like grass, foliage, water, and so forth. Moreover, edges due to texture are often stronger than edges due to object contours. This implies that further processing is needed to discriminate object contours from texture edges. In this paper, we propose a biologically motivated multiresolution contour detection method using Bayesian denoising and a surround inhibition technique. Specifically, the proposed approach deploys computation of the gradient at different resolutions, followed by Bayesian denoising of the edge image. Then, a biologically motivated surround inhibition step is applied in order to suppress edges that are due to texture. We propose an improvement of the surround suppression used in previous works. Finally, a contour-oriented binarization algorithm is used, relying on the observation that object contours lead to long connected components rather than to short rods obtained from textures. Experimental results show that our contour detection method outperforms standard edge detectors as well as other methods that deploy inhibition.

Local luminance amplitude modulates the interpretation of shape-from-shading in textured surfaces

The pattern of illumination on an undulating surface can be used to infer its 3-D form (shape-from-shading). But the recovery of shape would be invalid if the luminance changes actually arose from changes in reflectance. So how does vision distinguish variation in illumination from variation in reflectance to avoid illusory depth? When a corrugated surface is painted with an albedo texture, the variation in local mean luminance (LM) due to shading is accompanied by a similar modulation in local luminance amplitude (AM). This is not so for reflectance variation, nor for roughly textured surfaces. We used depth mapping and paired comparison methods to show that modulations of local luminance amplitude play a role in the interpretation of shape-from-shading. The shape-from-shading percept was enhanced when LM and AM co-varied (in-phase) and was disrupted when they were out of phase or (to a lesser degree) when AM was absent. The perceptual differences between cue types (in-phase vs out-of-phase) were enhanced when the two cues were present at different orientations within a single image. Our results suggest that when LM and AM co-vary (in-phase) this indicates that the source of variation is illumination (caused by undulations of the surface), rather than surface reflectance. Hence, the congruence of LM and AM is a cue that supports a shape-from-shading interpretation.

A feature-tracking model simulates the motion direction bias induced by phase congruency

Here we report a new motion illusion where the prevailing motion direction is strongly influenced by the relative phase of the harmonic components of the stimulus. The basic stimulus is the sum of three sinusoidal contrast-reversing gratings: the first, the third, and the fifth harmonic of two square wave gratings that drift in opposite direction. The phase of one of the fifth components was kept constant at 180 deg, whereas the phase of the other fifth harmonic was varied over the range 0-150 deg. For each phase value of the fifth harmonic, the motion was strongly biased toward its direction, corresponding to the direction with stronger phase congruency between the three harmonics. The strength of the prevailing motion was assessed by measuring motion direction discrimination thresholds, by varying the contrast of the third and the fifth harmonics plaid pattern. Results show that the contrast of high harmonics had to be increased by more than a factor of 10, to achieve a balance of motion for phase differences greater than 60 deg between the 2 fifth harmonics. We also measured the dependence on the absolute phase of harmonic components and found that it is not an important parameter, excluding the possibility that local luminance cues could be mediating the effect. A feature-tracking model based on previous work is proposed to simulate the data. The model computes local energy function from a pair of space-time separable front stage filters and applies a battery of directional second stage mechanisms. It is able to simulate quantitatively the phase congruency dependence illusion and the insensitivity to overall phase. Other energy models based on directional filters fail to simulate the phase congruency dependency effect.

Glaucoma Detection Is Facilitated by Luminance Modulation of the Global Flash Multifocal Electroretinogram

To investigate the variation of retinal electrophysiological function in glaucoma by using the global flash multifocal electroretinogram (mfERG) stimulation with altered differences in the stimulus luminance of the multifocal flashes, in an attempt to alter the levels of inner retinal contributions. The mfERG was assessed with a visual stimulus in steps of four video frames, which consisted of 103 scaled hexagonal elements followed by a dark frame, global flash, and dark frame. The localized luminance difference was set at 96%, 65%, 49%, or 29% stimulus contrast. Thirty subjects with glaucoma and 30 age-matched normal subjects were recruited for visual field and mfERG measurements. This stimulus induces complex local first-order responses with an early direct component (DC) and a later induced component (IC). The luminance-modulated response functions of the DC and IC responses showed markedly different behavior. The peripheral IC showed a linear dependence on luminance difference, whereas the peripheral DC was saturated for higher luminance differences. This saturation became less obvious in subjects with glaucoma, mostly because of greater reduction of the response amplitude in the mid luminance-difference level. An "adaptive index" was calculated from the luminance-difference dependence of the peripheral DC, and it showed a sensitivity of 93%, with a specificity of 95% for differentiating normal from glaucomatous eyes, and also had a significant correlation (r = 0.58) with the glaucomatous visual field defect. The peripheral DC luminance-modulated response function is altered by the adaptive mechanism that is induced by the global flash; the reduction of the adaptive index may thus relate to an abnormal adaptive mechanism, presumably due to inner retinal damage. Glaucoma appears to produce large reductions of the adaptive index which correlate with field defects.

Lightness constancy in 4-month-old infants

In the present study, discrete trial familiarization/novelty techniques were used to study lightness constancy in 4-month-old infants. The test stimuli were real objects (paper smiley faces) of two different reflectances, dark gray (17% reflectance) and light gray (54% reflectance). In Experiment 1, the test stimuli were viewed against a white (90% reflectance) surround, and in Experiment 2, against a black (4.6% reflectance) surround. In Experiments 1 and 2, the illumination was changed between familiarization and test phases of each trial. In Experiment 3, the reflectance of the surround was changed from white to mid gray (28.5% reflectance) between familiarization and test phases of each trial. With the white surround, the infants preferred the face with the novel reflectance, consistent with the presence of lightness constancy. With the black surround, the infants showed no preference between faces with novel reflectance vs. novel luminance. With the changing surround, the infants showed a small preference for the stimulus with the novel ratio, as opposed to the stimulus with the novel reflectance and the novel luminance. The results are discussed in the context of adult cues for lightness constancy, including white anchor points and local luminance ratios.

Local luminance and contrast in natural images

Within natural images there is substantial spatial variation in both local contrast and local luminance. Understanding the statistics of these variations is important for understanding the dynamics of receptive field stimulation that occur under natural viewing conditions and for understanding the requirements for effective luminance and contrast gain control. Local luminance and contrast were measured in a large set of calibrated 12-bit gray-scale natural images, for a number of analysis patch sizes. For each image and patch size we measured the range of contrast, the range of luminance, the correlation in contrast and luminance as a function of the distance between patches, and the correlation between contrast and luminance within patches. The same analyses were also performed on hand segmented regions containing only “sky”, “ground”, “foliage”, or “backlit foliage”. Within the typical image, the 95% range (2.5–97.5 percentile) for both local luminance and local contrast is somewhat greater than a factor of 10. The correlation in contrast and the correlation in luminance diminish rapidly with distance, and the typical correlation between luminance and contrast within patches is small (e.g., −0.2 compared to −0.8 for 1/ f noise). We show that eye movements are frequently large enough that there will be little correlation in the contrast or luminance on a receptive field from one fixation to the next, and thus rapid contrast and luminance gain control are essential. The low correlation between local luminance and contrast implies that efficient contrast gain control mechanisms can operate largely independently of luminance gain control mechanisms.

Independence of luminance and contrast in natural scenes and in the early visual system

The early visual system is endowed with adaptive mechanisms that rapidly adjust gain and integration time based on the local luminance (mean intensity) and contrast (standard deviation of intensity relative to the mean). Here we show that these mechanisms are matched to the statistics of the environment. First, we measured the joint distribution of luminance and contrast in patches selected from natural images and found that luminance and contrast were statistically independent of each other. This independence did not hold for artificial images with matched spectral characteristics. Second, we characterized the effects of the adaptive mechanisms in lateral geniculate nucleus (LGN), the direct recipient of retinal outputs. We found that luminance gain control had the same effect at all contrasts and that contrast gain control had the same effect at all mean luminances. Thus, the adaptive mechanisms for luminance and contrast operate independently, reflecting the very independence encountered in natural images.

Do New Objects Capture Attention?

The visual system relies on several heuristics to direct attention to important locations and objects. One of these mechanisms directs attention to sudden changes in the environment. Although a substantial body of research suggests that this capture of attention occurs only for the abrupt appearance of a new perceptual object, more recent evidence shows that some luminance-based transients (e.g., motion and looming) and some types of brightness change also capture attention. These findings show that new objects are not necessary for attention capture. The present study tested whether they are even sufficient. That is, does a new object attract attention because the visual system is sensitive to new objects or because it is sensitive to the transients that new objects create? In two experiments using a visual search task, new objects did not capture attention unless they created a strong local luminance transient.

Interactions between luminance and contrast signals in global form detection

The human visual system is adept at detecting global structure, or form, within a scene. The initial stage of post-retinal processing for all aspects of vision is fed by On- and Off-centre cells sensitive to centred luminance increments and decrements respectively. These cells provide input to two parallel pathways that process variations in local luminance (first-order pathway) and local contrast (second-order pathway). Here, we investigate the contribution of luminance and contrast information to global form detection, a stage between the extraction of local orientation and the recognition of objects. The underlying processes involve two stages. We find that signals in the On-, Off- and second-order pathways are segregated at both stages of processing. Surprisingly, the non-linear stage in the second-order form pathway is different from that in motion processing: the second-order form detectors show an asymmetry in sensitivity to increments and decrements that is not apparent in motion. A functional architecture for global form detection is proposed along with its possible neural substrates.

Perception of Gaze Direction Based on Luminance Ratio

Changing the luminance of one side of the sclera induces an apparent shift of the perceived direction of gaze toward the darker side of the sclera (Ando, 2002 Perception 31 657-674). However, when both the sclera and the skin surrounding it were darkened simultaneously as if by a cast shadow, the apparent direction of gaze shifted less than when only the sclera was darkened. The results suggest that one mechanism for gaze judgment is based on a simple analysis of the local luminance ratio between the eye and the surrounding region.

The statistics of local contrast and mean luminance in natural images

The statistics of local contrast and mean luminance in natural images

Beyond fourth-order texture discrimination: generation of extreme-order and statistically-balanced textures

Julesz [IRE Trans. Inf. Theory IT-8 (1962) 84] introduced the concept of statistically defined textures and their perceptual discrimination. Julesz [Sci. Am. 232 (1975) 34] showed that discrimination was possible with statistics equated to third-order, specifying fourth-order textures. Klein and Tyler [J. Opt. Soc. Am. A 3 (1986) 868] offered a variety of paradigms suggesting that fourth order might be the limit on human texture processing. To go beyond this limit, new texture paradigms are now introduced to avoid contamination by luminance extrema, to control local and long-range texture properties, and to provide textures without global statistical structure. Local luminance contamination is avoided by novel orientation plaids, in which higher-order rules govern the orientation of local elements rather than their coloring. These textures allow evaluation of texture discrimination up to thirty-second order by cortical pattern elements. Long-range processing is studied by random strip rotation and by interlacing of independent textures. Each substantially degrades the visibility of the fourth-order textures, revealing that the fourth-order information is conveyed largely by local rather than long-range perturbations from random statistics. Finally, textures equated at all orders can be defined in terms of their global statistics, but may nevertheless readily be discriminated in human vision. The discrimination on the basis of local perturbations implies that human vision assesses textures through a local sampling window, and is largely insensitive to longer-range statistical properties.