Surface Filling-in Research Articles

Attentional selection and illusory surface appearance

The visual system is required to compute objects from partial image structure so that figures can be segmented from their backgrounds. Although early clinical, behavioral, and modeling data suggested that such computations are performed pre-attentively, recent neurophysiological evidence suggests that surface filling-in is influenced by attention. In the present study we developed a variant of the classical Kanizsa illusory triangle to investigate whether voluntary attention modulates perceptual filling-in. Our figure consists of “pacmen” positioned at the tips of an illusory 6-point star and alternating in polarity such that two illusory triangles are implied to compete with one another within the figure. On each trial, observers were cued to attend to only one triangle, and then compared its lightness with a matching texture-defined triangle. We found that perceived lightness of the illusory shape depended on the polarity of pacmen framing the attended triangle. Our findings thus reveal that, for overlapping illusory surfaces, lightness judgements can depend on voluntary attention. Our novel stimulus may prove useful in future attempts to link neurophysiological effects to phenomenology.

Surface filling-in and contour interpolation contribute independently to Kanizsa figure formation.

To explore mechanisms of object integration, the present experiments examined how completion of illusory contours and surfaces modulates the sensitivity of localizing a target probe. Observers had to judge whether a briefly presented dot probe was located inside or outside the region demarcated by inducer elements that grouped to form variants of an illusory, Kanizsa-type figure. From the resulting psychometric functions, we determined observers' discrimination thresholds as a sensitivity measure. Experiment 1 showed that sensitivity was systematically modulated by the amount of surface and contour completion afforded by a given configuration. Experiments 2 and 3 presented stimulus variants that induced an (occluded) object without clearly defined bounding contours, which gave rise to a relative sensitivity increase for surface variations on their own. Experiments 4 and 5 were performed to rule out that these performance modulations were simply attributable to variable distances between critical local inducers or to costs in processing an interrupted contour. Collectively, the findings provide evidence for a dissociation between surface and contour processing, supporting a model of object integration in which completion is instantiated by feedforward processing that independently renders surface filling-in and contour interpolation and a feedback loop that integrates these outputs into a complete whole. (PsycINFO Database Record

Supplemental Material for Surface Filling-In and Contour Interpolation Contribute Independently to Kanizsa Figure Formation

Supplemental Material for Surface Filling-In and Contour Interpolation Contribute Independently to Kanizsa Figure Formation

Cortical Dynamics of Figure-Ground Separation in Response to 2D Pictures and 3D Scenes: How V2 Combines Border Ownership, Stereoscopic Cues, and Gestalt Grouping Rules.

The FACADE model, and its laminar cortical realization and extension in the 3D LAMINART model, have explained, simulated, and predicted many perceptual and neurobiological data about how the visual cortex carries out 3D vision and figure-ground perception, and how these cortical mechanisms enable 2D pictures to generate 3D percepts of occluding and occluded objects. In particular, these models have proposed how border ownership occurs, but have not yet explicitly explained the correlation between multiple properties of border ownership neurons in cortical area V2 that were reported in a remarkable series of neurophysiological experiments by von der Heydt and his colleagues; namely, border ownership, contrast preference, binocular stereoscopic information, selectivity for side-of-figure, Gestalt rules, and strength of attentional modulation, as well as the time course during which such properties arise. This article shows how, by combining 3D LAMINART properties that were discovered in two parallel streams of research, a unified explanation of these properties emerges. This explanation proposes, moreover, how these properties contribute to the generation of consciously seen 3D surfaces. The first research stream models how processes like 3D boundary grouping and surface filling-in interact in multiple stages within and between the V1 interblob—V2 interstripe—V4 cortical stream and the V1 blob—V2 thin stripe—V4 cortical stream, respectively. Of particular importance for understanding figure-ground separation is how these cortical interactions convert computationally complementary boundary and surface mechanisms into a consistent conscious percept, including the critical use of surface contour feedback signals from surface representations in V2 thin stripes to boundary representations in V2 interstripes. Remarkably, key figure-ground properties emerge from these feedback interactions. The second research stream shows how cells that compute absolute disparity in cortical area V1 are transformed into cells that compute relative disparity in cortical area V2. Relative disparity is a more invariant measure of an object's depth and 3D shape, and is sensitive to figure-ground properties.

Serial versus parallel processing in mid-level vision: filling-in the details of spatial interpolation.

The relationship between boundary completion and surface filling-in, two core mechanisms of mid-level vision, remains unclear. Here, we integrate recent empirical findings to shine new light onto the neural mechanisms of boundary completion and surface filling-in as well as their relation to each other. Specifically, we discuss several psychophysical and neurophysiological studies that, when taken together, support a model where object boundaries and visual surfaces are interpolated in parallel, with one process impacting the other. We suggest that visual boundary completion and surface filling-in are two interacting processes that are supported by neural processes that are distributed throughout several areas of the early visual system.

How the venetian blind percept emerges from the laminar cortical dynamics of 3D vision.

The 3D LAMINART model of 3D vision and figure-ground perception is used to explain and simulate a key example of the Venetian blind effect and to show how it is related to other well-known perceptual phenomena such as Panum's limiting case. The model proposes how lateral geniculate nucleus (LGN) and hierarchically organized laminar circuits in cortical areas V1, V2, and V4 interact to control processes of 3D boundary formation and surface filling-in that simulate many properties of 3D vision percepts, notably consciously seen surface percepts, which are predicted to arise when filled-in surface representations are integrated into surface-shroud resonances between visual and parietal cortex. Interactions between layers 4, 3B, and 2/3 in V1 and V2 carry out stereopsis and 3D boundary formation. Both binocular and monocular information combine to form 3D boundary and surface representations. Surface contour surface-to-boundary feedback from V2 thin stripes to V2 pale stripes combines computationally complementary boundary and surface formation properties, leading to a single consistent percept, while also eliminating redundant 3D boundaries, and triggering figure-ground perception. False binocular boundary matches are eliminated by Gestalt grouping properties during boundary formation. In particular, a disparity filter, which helps to solve the Correspondence Problem by eliminating false matches, is predicted to be realized as part of the boundary grouping process in layer 2/3 of cortical area V2. The model has been used to simulate the consciously seen 3D surface percepts in 18 psychophysical experiments. These percepts include the Venetian blind effect, Panum's limiting case, contrast variations of dichoptic masking and the correspondence problem, the effect of interocular contrast differences on stereoacuity, stereopsis with polarity-reversed stereograms, da Vinci stereopsis, and perceptual closure. These model mechanisms have also simulated properties of 3D neon color spreading, binocular rivalry, 3D Necker cube, and many examples of 3D figure-ground separation.

Stereopsis and 3D surface perception by spiking neurons in laminar cortical circuits: A method for converting neural rate models into spiking models

A laminar cortical model of stereopsis and 3D surface perception is developed and simulated. The model shows how spiking neurons that interact in hierarchically organized laminar circuits of the visual cortex can generate analog properties of 3D visual percepts. The model describes how monocular and binocular oriented filtering interact with later stages of 3D boundary formation and surface filling-in in the LGN and cortical areas V1, V2, and V4. It proposes how interactions between layers 4, 3B, and 2/3 in V1 and V2 contribute to stereopsis, and how binocular and monocular information combine to form 3D boundary and surface representations. The model suggests how surface-to-boundary feedback from V2 thin stripes to pale stripes helps to explain how computationally complementary boundary and surface formation properties lead to a single consistent percept, eliminate redundant 3D boundaries, and trigger figure-ground perception. The model also shows how false binocular boundary matches may be eliminated by Gestalt grouping properties. In particular, the disparity filter, which helps to solve the correspondence problem by eliminating false matches, is realized using inhibitory interneurons as part of the perceptual grouping process by horizontal connections in layer 2/3 of cortical area V2. The 3D sLAMINART model simulates 3D surface percepts that are consciously seen in 18 psychophysical experiments. These percepts include contrast variations of dichoptic masking and the correspondence problem, the effect of interocular contrast differences on stereoacuity, Panum's limiting case, the Venetian blind illusion, stereopsis with polarity-reversed stereograms, da Vinci stereopsis, and perceptual closure. The model hereby illustrates a general method of unlumping rate-based models that use the membrane equations of neurophysiology into models that use spiking neurons, and which may be embodied in VLSI chips that use spiking neurons to minimize heat production.

Rebound Spiking as a Neural Mechanism for Surface Filling-in

Perceptual filling-in is the phenomenon where visual information is perceived although information is not physically present. For instance, the blind spot, which corresponds to the retinal location where there are no photoreceptor cells to capture the visual signals, is filled-in by the surrounding visual signals. The neural mechanism for such immediate filling-in of surfaces is unclear. By means of computational modeling, we show that surround inhibition produces rebound or after-discharge spiking in neurons that otherwise do not receive sensory information. The behavior of rebound spiking mimics the immediate surface filling-in illusion observed at the blind spot and also reproduces the filling-in of an empty object after a background flash, like in the color dove illusion. In conclusion, we propose rebound spiking as a possible neural mechanism for surface filling-in.

From stereogram to surface: how the brain sees the world in depth

When we look at a scene, how do we consciously see surfaces infused with lightness and color at the correct depths? Random-Dot Stereograms (RDS) probe how binocular disparity between the two eyes can generate such conscious surface percepts. Dense RDS do so despite the fact that they include multiple false binocular matches. Sparse stereograms do so even across large contrast-free regions with no binocular matches. Stereograms that define occluding and occluded surfaces lead to surface percepts wherein partially occluded textured surfaces are completed behind occluding textured surfaces at a spatial scale much larger than that of the texture elements themselves. Earlier models suggest how the brain detects binocular disparity, but not how RDS generate conscious percepts of 3D surfaces. This article proposes a neural network model that predicts how the layered circuits of visual cortex generate these 3D surface percepts using interactions between visual boundary and surface representations that obey complementary computational rules. The model clarifies how interactions between layers 4, 3B and 2/3A in V1 and V2 contribute to stereopsis, and proposes how 3D perceptual grouping laws in V2 interact with 3D surface filling-in operations in V1, V2 and V4 to generate 3D surface percepts in which figures are separated from their backgrounds. The model explanations of 3D surface percepts raised by various RDS are demonstrated by computer simulations. The model hereby unifies the explanation of data about stereopsis and data about 3D figure-ground separation and completion of partially occluded object surfaces. It shows how these model mechanisms convert the complementary rules for boundary and surface formation into consistent visual percepts of 3D surfaces.

Preattentive surface and contour grouping in Kanizsa figures: Evidence from parietal extinction

Visual extinction commonly occurs after unilateral, parietal brain damage and manifests in a failure to identify contralesional stimuli when presented simultaneously with other, ipsilesional stimuli – but full awareness for single stimulus presentations. However, extinction can be substantially reduced when preattentive grouping operations link fragmentary items across hemifields into a coherent object. For instance, one study demonstrated preserved access to bilateral stimulus segments when these could be grouped to form a Kanizsa square [Mattingley, J. B., Davis, G., & Driver, J. (1997). Preattentive filling-in of visual surfaces in parietal extinction. Science, 275, 671–674]. Here, we investigated the relative contributions of distinct object attributes to the spared access in Kanizsa figure completion in extinction, by systematically varying the degree to which bilateral surface filling-in and contour interpolations group disparate items. We demonstrate that surface information can substantially reduce extinction, whereas contour completions showed comparably smaller influences. In summary, such graded influences of object attributes support recurrent models of grouping, first, linking fragmentary parts into coherent surfaces and, second, interpolating the precise boundaries.

V1 Response Timing and Surface Filling-In

There is ample evidence from demonstrations such as color induction and stabilized images that information from surface boundaries plays a special role in determining the perception of surface interiors. Surface interiors appear to "fill-in." Psychophysical experiments also show that surface perception involves a slow scale-dependent process distinct from mechanisms involved in contour perception. The present experiments aimed to test the hypothesis that surface perception is associated with relatively slow scale-dependent neural filling-in. We found that responses in macaque primary visual cortex (V1) are slower to surface interiors than responses to optimal bar stimuli. Moreover, we found that the response to a surface interior is delayed relative to the response to the surface's border and the extent of the delay is proportional to the distance between a receptive field and the border. These findings are consistent with some forms of neural filling-in and suggest that V1 may provide the neural substrate for perceptual filling-in.

The contrasting impact of global and local object attributes on Kanizsa figure detection

Studies on the involvement of object completions in search for illusory figures have so far reported equivocal results. We have addressed this issue by investigating at which level object attributes in Kanizsa figures influence search. Employing a paradigm that investigated global and local attributes in the composition of distractors with relation to target composition, we report a selective involvement of multilevel processing upon detection. Four experiments demonstrate that global surface information, but not the surrounding global contour, determines the speed of Kanizsa figure detection. By contrast, local inducer information is encoded far less efficiently in search than processes computing the global object. Our conclusions are that surface filling-in acts as a major determinant of search, but depends on the relevance of the particular hierarchical level (local or global) coding the target.

Texture segregation by visual cortex: Perceptual grouping, attention, and learning

A neural model called dARTEX is proposed of how laminar interactions in the visual cortex may learn and recognize object texture and form boundaries. The model unifies five interacting processes: region-based texture classification, contour-based boundary grouping, surface filling-in, spatial attention, and object attention. The model shows how form boundaries can determine regions in which surface filling-in occurs; how surface filling-in interacts with spatial attention to generate a form-fitting distribution of spatial attention, or attentional shroud; how the strongest shroud can inhibit weaker shrouds; and how the winning shroud regulates learning of texture categories, and thus the allocation of object attention. The model can discriminate abutted textures with blurred boundaries and is sensitive to texture boundary attributes like discontinuities in orientation and texture flow curvature as well as to relative orientations of texture elements. The model quantitatively fits the Ben-Shahar and Zucker [Ben-Shahar, O. & Zucker, S. (2004). Sensitivity to curvatures in orientation-based texture segmentation. Vision Research, 44, 257–277] human psychophysical data on orientation-based textures. Surface-based attentional shrouds improve texture learning and classification: Brodatz texture classification rate varies from 95.1% to 98.6% with correct attention, and from 74.1% to 75.5% without attention. Object boundary output of the model in response to photographic images is compared to computer vision algorithms and human segmentations.

A neural model of 3D shape-from-texture: Multiple-scale filtering, boundary grouping, and surface filling-in

A neural model is presented of how cortical areas V1, V2, and V4 interact to convert a textured 2D image into a representation of curved 3D shape. Two basic problems are solved to achieve this: (1) Patterns of spatially discrete 2D texture elements are transformed into a spatially smooth surface representation of 3D shape. (2) Changes in the statistical properties of texture elements across space induce the perceived 3D shape of this surface representation. This is achieved in the model through multiple-scale filtering of a 2D image, followed by a cooperative-competitive grouping network that coherently binds texture elements into boundary webs at the appropriate depths using a scale-to-depth map and a subsequent depth competition stage. These boundary webs then gate filling-in of surface lightness signals in order to form a smooth 3D surface percept. The model quantitatively simulates challenging psychophysical data about perception of prolate ellipsoids [Todd, J., & Akerstrom, R. (1987). Perception of three-dimensional form from patterns of optical texture. Journal of Experimental Psychology: Human Perception and Performance, 13(2), 242–255]. In particular, the model represents a high degree of 3D curvature for a certain class of images, all of whose texture elements have the same degree of optical compression, in accordance with percepts of human observers. Simulations of 3D percepts of an elliptical cylinder, a slanted plane, and a photo of a golf ball are also presented.

A neural model of 3D shape-from-texture: Multiple-scale filtering, cooperative-competitive grouping, and 3D surface filling-in

A neural model of 3D shape-from-texture: Multiple-scale filtering, cooperative-competitive grouping, and 3D surface filling-in

A laminar cortical model of stereopsis and 3D surface perception: closure and da Vinci stereopsis

A laminar cortical model of stereopsis and 3D surface perception is developed and simulated. The model describes how monocular and binocular oriented filtering interact with later stages of 3D boundary formation and surface filling-in in the LGN and cortical areas V1, V2, and V4. It proposes how interactions between layers 4, 3B, and 2/3 in V1 and V2 contribute to stereopsis, and how binocular and monocular information combine to form 3D boundary and surface representations. The model includes two main new developments: (1) It clarifies how surface-to-boundary feedback from V2 thin stripes to pale stripes helps to explain data about stereopsis. This feedback has previously been used to explain data about 3D figure-ground perception. (2) It proposes that the binocular false match problem is subsumed under the Gestalt grouping problem. In particular, the disparity filter, which helps to solve the correspondence problem by eliminating false matches, is realized using inhibitory interneurons as part of the perceptual grouping process by horizontal connections in layer 2/3 of cortical area V2. The enhanced model explains all the psychophysical data previously simulated by Grossberg and Howe (2003), such as contrast variations of dichoptic masking and the correspondence problem, the effect of interocular contrast differences on stereoacuity, Panum's limiting case, the Venetian blind illusion, stereopsis with polarity-reversed stereograms, and da Vinci stereopsis. It also explains psychophysical data about perceptual closure and variations of da Vinci stereopsis that previous models cannot yet explain.

A laminar cortical model of 3D shape-from-texture: spatial-scale filtering, cooperative-competitive grouping, and surface filling-in

A laminar cortical model of 3D shape-from-texture: spatial-scale filtering, cooperative-competitive grouping, and surface filling-in

A laminar cortical model of stereopsis and three-dimensional surface perception

A laminar cortical model of stereopsis and later stages of 3D surface perception is developed and simulated. The model describes how initial stages of monocular and binocular oriented filtering interact with later stages of 3D boundary formation and surface filling-in in the lateral geniculate nucleus and cortical areas V1, V2, and V4. In particular, it details how interactions between layers 4, 3B, and 2/3A in V1 and V2 contribute to stereopsis, and clarifies how binocular and monocular information combine to form 3D boundary and surface representations. Along the way, the model modifies and significantly extends the disparity energy model. Neural explanations are given for psychophysical data concerning: contrast variations of dichoptic masking and the correspondence problem, the effect of interocular contrast differences on stereoacuity, Panum’s limiting case, the Venetian blind illusion, stereopsis with polarity-reversed stereograms, da Vinci stereopsis, and various lightness illusions. By relating physiology to psychophysics, the model provides new functional insights and predictions about laminar cortical architecture.

A contour propagation approach to surface filling-in and volume formation.

A new approach to surface and volume formation is introduced in response to the question, "Why do some silhouettes look 3 dimensional (3D) and others look 2D?" The central idea is that form information can propagate away from a "propagable segment" (PS) of occluding contour that could have projected onto the image from the visible portion of a cross-section of a surface. A key property of a PS is that it exhibits abrupt curvature changes where it meets the rest of the occluding contour. An algorithm is described for filling in curved surfaces from a PS: When copies of a PS are propagated into the interior, they act as cross-sectional surface contours that also exhibit abrupt curvature changes with the rest of the occluding contour. The result is a nonmetric coding of 3D-shape in terms of local ordinal surface curvature and orientation relationships that is scale, translation, and rotation invariant.

A contour propagation approach to surface filling-in and volume formation.

A new approach to surface and volume formation is introduced in response to the question, Why do some silhouettes look 3 dimensional (3D) and others look 2D? The central idea is that form information can propagate away from a propagable segment (PS) of occluding contour that could have projected onto the image from the visible portion of a cross-section of a surface. A key property of a PS is that it exhibits abrupt curvature changes where it meets the rest of the occluding contour. An algorithm is described for filling in curved surfaces from a PS: When copies of a PS are propagated into the interior, they act as cross-sectional surface contours that also exhibit abrupt curvature changes with the rest of the occluding contour. The result is a nonmetric coding of 3D-shape in terms of local ordinal surface curvature and orientation relationships that is scale, translation, and rotation invariant.