Filling-in Phenomenon Research Articles

Illusory sound texture reveals multi-second statistical completion in auditory scene analysis

Sound sources in the world are experienced as stable even when intermittently obscured, implying perceptual completion mechanisms that “fill in” missing sensory information. We demonstrate a filling-in phenomenon in which the brain extrapolates the statistics of background sounds (textures) over periods of several seconds when they are interrupted by another sound, producing vivid percepts of illusory texture. The effect differs from previously described completion effects in that 1) the extrapolated sound must be defined statistically given the stochastic nature of texture, and 2) the effect lasts much longer, enabling introspection and facilitating assessment of the underlying representation. Illusory texture biases subsequent texture statistic estimates indistinguishably from actual texture, suggesting that it is represented similarly to actual texture. The illusion appears to represent an inference about whether the background is likely to continue during concurrent sounds, providing a stable statistical representation of the ongoing environment despite unstable sensory evidence.

Predictive Coding: A Possible Explanation of Filling-In at the Blind Spot.

Filling-in at the blind spot is a perceptual phenomenon in which the visual system fills the informational void, which arises due to the absence of retinal input corresponding to the optic disc, with surrounding visual attributes. It is known that during filling-in, nonlinear neural responses are observed in the early visual area that correlates with the perception, but the knowledge of underlying neural mechanism for filling-in at the blind spot is far from complete. In this work, we attempted to present a fresh perspective on the computational mechanism of filling-in process in the framework of hierarchical predictive coding, which provides a functional explanation for a range of neural responses in the cortex. We simulated a three-level hierarchical network and observe its response while stimulating the network with different bar stimulus across the blind spot. We find that the predictive-estimator neurons that represent blind spot in primary visual cortex exhibit elevated non-linear response when the bar stimulated both sides of the blind spot. Using generative model, we also show that these responses represent the filling-in completion. All these results are consistent with the finding of psychophysical and physiological studies. In this study, we also demonstrate that the tolerance in filling-in qualitatively matches with the experimental findings related to non-aligned bars. We discuss this phenomenon in the predictive coding paradigm and show that all our results could be explained by taking into account the efficient coding of natural images along with feedback and feed-forward connections that allow priors and predictions to co-evolve to arrive at the best prediction. These results suggest that the filling-in process could be a manifestation of the general computational principle of hierarchical predictive coding of natural images.

Experience-based Auditory Predictions Modulate Brain Activity to Silence as do Real Sounds.

Interactions between stimuli's acoustic features and experience-based internal models of the environment enable listeners to compensate for the disruptions in auditory streams that are regularly encountered in noisy environments. However, whether auditory gaps are filled in predictively or restored a posteriori remains unclear. The current lack of positive statistical evidence that internal models can actually shape brain activity as would real sounds precludes accepting predictive accounts of filling-in phenomenon. We investigated the neurophysiological effects of internal models by testing whether single-trial electrophysiological responses to omitted sounds in a rule-based sequence of tones with varying pitch could be decoded from the responses to real sounds and by analyzing the ERPs to the omissions with data-driven electrical neuroimaging methods. The decoding of the brain responses to different expected, but omitted, tones in both passive and active listening conditions was above chance based on the responses to the real sound in active listening conditions. Topographic ERP analyses and electrical source estimations revealed that, in the absence of any stimulation, experience-based internal models elicit an electrophysiological activity different from noise and that the temporal dynamics of this activity depend on attention. We further found that the expected change in pitch direction of omitted tones modulated the activity of left posterior temporal areas 140-200 msec after the onset of omissions. Collectively, our results indicate that, even in the absence of any stimulation, internal models modulate brain activity as do real sounds, indicating that auditory filling in can be accounted for by predictive activity.

Contributions of contour frequency, amplitude, and luminance to the watercolor effect estimated by conjoint measurement

The watercolor effect is a long-range, assimilative, filling-in phenomenon induced by a pair of distant, wavy contours of different chromaticities. Here, we measured joint influences of the contour frequency and amplitude and the luminance of the interior contour on the strength of the effect. Contour pairs, each enclosing a circular region, were presented with two of the dimensions varying independently across trials (luminance/frequency, luminance/amplitude, frequency/amplitude) in a conjoint measurement paradigm (Luce & Tukey, 1964). In each trial, observers judged which of the stimuli evoked the strongest fill-in color. Control stimuli were identical except that the contours were intertwined and generated little filling-in. Perceptual scales were estimated by a maximum likelihood method (Ho, Landy, & Maloney, 2008). An additive model accounted for the joint contributions of any pair of dimensions. As shown previously using difference scaling (Devinck & Knoblauch, 2012), the strength increases with luminance of the interior contour. The strength of the phenomenon was nearly independent of the amplitude of modulation of the contour but increased with its frequency up to an asymptotic level. On average, the strength of the effect was similar along a given dimension regardless of the other dimension with which it was paired, demonstrating consistency of the underlying estimated perceptual scales.

Perception of illusory contours forms intermodulation responses of steady state visual evoked potentials as a neural signature of spatial integration

Perception of illusory contours was shown to be a consequence of neural activity related to spatial integration in early visual areas. Candidates for such filling-in phenomena are long-range horizontal connections of neurons in V1/V2, and feedback from higher order visual areas. To get a direct measure of spatial integration in early visual cortex, we presented two differently flickering inducers, which evoked steady-state visual evoked potentials (SSVEPs) while manipulating the formation of an illusory rectangle. As a neural marker of integration we tested differences in amplitudes of intermodulation frequencies i.e. linear combinations of the driving frequencies. These were significantly increased when an illusory rectangle was perceived. Increases were neither due to changes of any of the two driving frequencies nor in the frequency that tagged the processing of the compound object, indicating that results are not a consequence of paying more attention to inducers when the illusory rectangle was visible.

A common signal detection model accounts for both perception and discrimination of the watercolor effect

Establishing the relation between perception and discrimination is a fundamental objective in psychophysics, with the goal of characterizing the neural mechanisms mediating perception. Here, we show that a procedure for estimating a perceptual scale based on a signal detection model also predicts discrimination performance. We use a recently developed procedure, Maximum Likelihood Difference Scaling (MLDS), to measure the perceptual strength of a long-range, color, filling-in phenomenon, the Watercolor Effect (WCE), as a function of the luminance ratio between the two components of its generating contour. MLDS is based on an equal-variance, gaussian, signal detection model and yields a perceptual scale with interval properties. The strength of the fill-in percept increased 10-15 times the estimate of the internal noise level for a 3-fold increase in the luminance ratio. Each observer's estimated scale predicted discrimination performance in a subsequent paired-comparison task. A common signal detection model accounts for both the appearance and discrimination data. Since signal detection theory provides a common metric for relating discrimination performance and neural response, the results have implications for comparing perceptual and neural response functions.

The effect of contrast intensity and polarity in the achromatic watercolor effect

The watercolor effect (WCE) is a filling-in phenomenon in a region demarcated by two thin abutting lines. The perceived chromaticity of the region is similar to that of the interior line. We develop a series of achromatic WCE stimuli to induce lightness changes analogous to the induced chromaticity in the chromatic version of the WCE. We use a variation of the paired-comparison paradigm to quantify the induced lightness of the filled-in regions to regions with real luminance variations. The luminance of the inner line is fixed, while the luminance of the outer line varies across stimuli. Data from seven subjects (five naive) confirm that an achromatic WCE exists. Moreover, outer lines with both high and low luminances can generate a WCE with an inner line of a moderate luminance. All subjects show a single peak of the effect strength for both polarity conditions, which is never at the extreme luminance levels. Most subjects show an inverted U curve for effect strength as a function of the contrast of the outer lines against the background. Results suggest that the contrast difference between the outer line and the inner line affects the existence and the strength of the achromatic WCE in a nonlinear way.

Thermal referral: A filling-in phenomenon that involves cross-modal processing of thermal and tactile stimuli

Thermal referral: A filling-in phenomenon that involves cross-modal processing of thermal and tactile stimuli

Modeling filling-in of afterimages

Van Lier, Vergeer, and Anstis (2009) reported that color information in a visual afterimage could spread across regions that were not colored in the inducing stimulus. The perceived color and shape of the afterimage could be manipulated by drawn contours that apparently trap the spread of afterimage color signals. They further hypothesized that the observed effects indicated a common mechanism for afterimage color filling-in and real-color filling-in phenomena. New simulations of the existing boundary contour system/feature contour system model of visual perception (Grossberg & Mingolla, 1985a, 1985b) demonstrate the connection between these phenomena.

Metacontrast masking of target-area internal contours and target overall surface brightness: the case of mutually coherent and incoherent visual objects

Metacontrast masking is by no means a unitary phenomenon, as is evidenced in recent studies showing differences between masking of surface- and contour properties of target stimuli (Breitmeyer et al., 2006; Ishikawa et al., 2006). Optima of masking appear earlier for contour processing and feature-specific operations compared to the variety of brightness processing that shows up in area filling-in phenomena. The present study explored whether this rule of processing - contours first and area filling-in afterwards - will be sustained if target and mask are, respectively, a central and a peripheral part of a coherent or incoherent meaningful visual object. Observers were presented with gray-level targets (images of the central part of a visual object) that were masked by a following, spatially surrounding mask, which was a complementary part of that object. Consistently with earlier findings, it appeared that salient visibility of contours which belonged to the internal spatial area of the target part of the object was established earlier and the whole-surface brightness quality (i.e. gray level) later in the course of target microgenesis. The unexpected facilitative effect of within-object coherence on target visibility which appeared at longer stimuli onset asynchrony (SOA) between target and mask parts of the object and only with large target and mask supports either some bias effects or lateral facilitatory interaction between iso-oriented parts of target-mask configuration having long time constants. The absence of the effects of coherence and inversion of target-plus-mask composite with small stimuli does not support the reentrant, top-down accounts of object processing in the context of metacontrast interactions.

Dynamic texture spreading: probing the mechanisms of surface interpolation

Dynamic texture spreading is a filling-in phenomenon where a colored pattern perceptually spreads onto an area confined by virtual contours in a multi-aperture motion display. The spreading effect is qualitatively similar to static texture spreading but widely surpasses it in strength, making it particularly suited for quantitative studies of visual interpolation processes. We first carried out two experiments to establish with objective tasks that texture spreading is a genuine representation of surface qualities and thus goes beyond mere contour interpolation. Two subsequent experiments serve to relate the phenomenon to ongoing discussions about potentially responsible mechanisms for spatiotemporal integration. With a phenomenological method, we examined to what extent simple sensory persistence might be causally involved in the effect under consideration. Most of our findings are consistent with the idea of sensory persistence, and indicate that information fragments are integrated over a time window of about 100 to 180 ms to form a complete surface representation.

Adaptation au scotome central : partie II: Complétion visuelle des scotomes centraux

The visual cortex may reorganize after occurrence of a scotoma. Different experimental studies have shown that there is an attempt to minimize the impact of the scotoma, the missing information being filled-in by surrounding information. The clinical consequences of this filling-in phenomenon have been extensively studied by Safran and co-workers. The present review summarizes the current literature on this phenomenon and its clinical consequences. Furthermore, the authors present their own experience with the filling-in phenomenon in patients with age-related macular degeneration. Their study shows that, with few exceptions, the phenomenon only occurs in patients with bilateral central scotoma, in their better eye.

V1 mechanisms explain filling-in phenomena in texture perception and visual search

V1 mechanisms explain filling-in phenomena in texture perception and visual search

Filling-in phenomenon in patients with age-related macular degeneration: differences regarding uni- or bilaterality of central scotoma.

The purpose of this study was to explore the presence of the filling-in phenomenon in patients with uni- or bilateral central scotoma (CS) resulting from natural history or laser photocoagulation of choroidal neovascularization in age-related macular degeneration (AMD). Sixteen consecutive patients with unilateral CS and 14 patients with bilateral CS were assessed (44 eyes) with a scanning laser ophthalmoscope (SLO). Scotoma was delineated by scotometry with a point (1 degree x1 degree) moving radially from the periphery to the center of the lesion. In addition, patients underwent a line test, consisting of a horizontal line moving vertically and a vertical line moving horizontally, from the periphery to the center. The lines were longer than the macular lesion and were projected onto the retina. Patients were asked to indicate when the lines seemed interrupted. The perceptual filling-in phenomenon was considered to be present when limits of the perceived scotoma, determined by the line test, were smaller than those assessed by scotometry. In patients with bilateral CS, the results were analyzed to distinguish the less or more severely affected eye. In all eyes, the limits of the scotoma obtained with the scotometry test corresponded to the anatomic edges of the macular lesion. In patients with bilateral CS, the filling-in phenomenon was observed in 12 out of 14 (85%) less severely affected eyes, but only in one (7%) of their more severely affected eyes. In patients with unilateral CS, the phenomenon was observed in only one out of 16 (6%) eyes. These results suggest that the filling-in phenomenon mostly occurs in patients with bilateral central scotoma, and almost always in their less affected eye. Thus, it did usually not occur in an eye if the fellow eye was better.

Perceptual filling-in and the resonant binding of distributed cortical representations

Pessoa et al. (1998a) summarize a wide body of data suggesting that perceptual filling-in phenomena can be attributed to neural filling-in processes. However, they reject, on philosophical grounds, the hypothesis that filled-in representations in the brain are the immediate substrate of visual percepts. It is proposed in this commentary that resonant binding between distributed cortical areas may instead be the crucial ingredient for conscious visual percepts, and that filling-in processes may facilitate the interactions between behaving organisms and object surfaces. These suggestions circumvent some of the philosophical problems associated with the idea of localized visual representations.

We can't fill in answers to philosophical questions

The target article discusses the classic blind spot, scotomas, subjective contours, and other so-called filling-in phenomena. Its purpose is to evaluate the idea that some theories of filling-in amount to tacit acceptance of Cartesian materialism and a form of psychophysical isomorphism. Pessoa et al. reject what is termed structural isomorphism as well as Cartesian materialism, but claim that neural processes adduced as underlying filling-in may be acceptable without implying isomorphism. The article supports the idea of perceiving as an active constructive process. However, the various subthemes are not clearly related to each other. Topological psychophysical isomorphism is indeed untenable, but the tacit assumption that filling-in enjoys any kind of unique status with respect to illuminating philosophical questions is doubtful.

Representations, computation, and inverse ecological optics

Implicit and explicit filling-in phenomena should be distinguished. Blind spot phenomena and mechanisms of boundary completion can be accounted for by implicit filling-in. Surface regions are “painted” with perceptual quantities, such as brightness, by explicit filling-in. “Filling-in” and “finding-out” relate to different computational tasks. Mechanisms of purposive computation (e.g., for navigation) evaluate local measurements, thus “finding out”; whereas mechanisms for grasping might require passive reconstruction, thus “filling in.”

The discrepancy between 99Tcm-ECD dynamic and static SPET images in patients with ischaemic lesions corresponds to reduced vasoreactivity to acetazolamide.

We evaluated 99Tcm-N,N'-(1,2-ethylenediyl)bis-L-cysteine diethyl ester (99Tcm-ECD) dynamic and static SPET (single photon emission tomographic) images to examine 99Tcm-ECD kinetics under ischaemic cerebrovascular conditions. In 20 patients who showed arterial occlusion on magnetic resonance angiography, dynamic (0-10 min) and static (15-35 min) SPET images were acquired after the intravenous administration of 99Tcm-ECD. Thirteen of the patients had focal perfusion deficits that were more evident on the dynamic than on the static images; the other seven showed no such discrepancy. In those patients with a mismatch between the dynamic and static images, the extent corresponded to reduced vaso-reactivity to acetazolamide. Based on quantitative analysis of the ratio of tracer uptake in affected to that in unaffected areas, the patients with discrepant findings showed significantly different ratios on the dynamic and static images, whereas those with no such mismatch did not. Our results suggest that dynamic 99Tcm-ECD images provide circulatory information and that static images reflect a filling-in phenomenon of ECD metabolites in ischaemic lesions. 99Tcm-ECD dynamic and static SPET images offer an alternative method of detecting mild perfusion deficits without the need for acetazolamide challenge.

Induced twinkle aftereffect as a probe of dynamic visual processing mechanisms

After viewing a blank patch surrounded by a dynamic noise stimulus (a video “snowstorm”), viewers report the prolonged perception of twinkle in the unstimulated region of the blank patch. We compare this induced twinkle aftereffect to the filling-in phenomenon, which may be seen in a small blank region, under similar test conditions but during stimulation. We found that strong induced twinkle aftereffects were seen both centrally and peripherally for blank test regions from 0.5 deg to as large as 20 deg in diameter, whereas filling-in was seen centrally only for test patch diameters smaller than 0.75 deg, becoming stronger peripherally but still limited to test regions less than about 3 deg in diameter. Lower noise density and larger noise element size facilitated filling-in but had little effect on the induced twinkle aftereflect. Conversely, noise frame rate had little effect on filling-in but had to be faster than 10 frames/sec to produce a twinkle aftereflect. Induced twinkle showed binocular superiority but no interocular transfer. The binocular superiority was partially explained by monocular blankout of the dynamic noise by the blank field in the occluded eye. These results all imply a different mechanism for the induced twinkle aftereffect than for filling-in. We consider a model in which the induced twinkle aftereflect is produced by post-inhibitory rebound in complex cells.

Color appearance of filled-in backgrounds affects hue cancellation, but not detection thresholds

A long-wavelength background can affect the appearance of an increment of light superimposed upon it in two ways. It can change the visual system's sensitivity to the increment, and it can change the appearance of the increment by directly adding redness to it. Through selective retinal-image stabilization, we evoked the filling-in phenomenon to change the appearance of 640- and 575-nm backgrounds. Either of these backgrounds could be made to appear red or yellow, depending upon whether it was viewed under stabilized or unstabilized conditions. When the appearance of the 640-nm background was altered by filling-in to appear less red, test probes superimposed upon it required less 540-nm component to achieve an equilibrium hue. Increment thresholds measured on the 640- and 575-nm backgrounds, however, did not change with the appearance of the backgrounds.