Non-retinotopic Processing Research Articles

Spatial congruency bias in identifying objects is triggered by retinal position congruence: Examination using the Ternus-Pikler illusion

When two different objects are sequentially presented at the same location, the viewer tends to misjudge them as identical (spatial congruency bias). The present study examined whether the spatial congruency bias would involve not only retinotopic but also non-retinotopic processing using the Ternus-Pikler illusion. In the experiments, two objects (central and peripheral) appeared in an initial frame. The target object was presented in the central area of the display, while the peripheral object was either on the left or right side of the target object. In the second frame, the target object was again presented in the central area, and the peripheral object was on the opposite side. Two kinds of inter-stimulus intervals were used. In the no-blank condition, the target object was perceived as stationary, and the peripheral object appeared to move to the opposite side. However, in the long-blank condition, the two objects were perceived to move together. Participants judged whether the target objects in the two frames were identical. As a result, the spatial congruency bias occurred irrespective of the ISI conditions. Our findings suggest that the spatial congruency bias is mainly based on retinotopic processing.

Non-retinotopic processing in humans versus retinotopic processing in pigeons(Summary of Awarded Presentation at the 31st Annual Meeting)

Non-retinotopic processing in humans versus retinotopic processing in pigeons(Summary of Awarded Presentation at the 31st Annual Meeting)

Retinotopic encoding of the Ternus-Pikler display reflected in the early visual areas.

The visual representation of the world is often assumed to be retinotopic, and many visual brain areas are indeed organized retinotopically. Visual perception, however, is not based on a reference frame anchored in retinotopic coordinates. For example, when an object moves, motion of its constituent parts is perceived relative to the object rather than in retinotopic coordinates. The moving object thus serves as a nonretinotopic reference system for computing the properties of its parts. It is largely unknown how the brain accomplishes this feat. Here, we used the Ternus-Pikler display to pit retinotopic processing in a stationary reference system against nonretinotopic processing in a moving one. Using 7T fMRI, we found that the average blood-oxygen-level dependent activations in V1, V2, and V3 reflected the retinotopic properties, but not the nonretinotopic percepts, of the Ternus-Pikler display. In the human motion processing complex (hMT+), activations were compatible with both retinotopic and nonretinotopic encoding. Thus, hMT+ may be the first visual area encoding the nonretinotopic percepts of the Ternus-Pikler display.

EEG Correlates of Relative Motion Encoding.

A large portion of the visual cortex is organized retinotopically, but perception is usually non-retinotopic. For example, a reflector on the spoke of a bicycle wheel appears to move on a circular or prolate cycloidal orbit as the bicycle moves forward, while in fact it traces out a curtate cycloidal trajectory. The moving bicycle serves as a non-retinotopic reference system to which the motion of the reflector is anchored. To study the neural correlates of non-retinotopic motion processing, we used the Ternus-Pikler display, where retinotopic processing in a stationary reference system is contrasted against non-retinotopic processing in a moving one. Using high-density EEG, we found similar brain responses for both retinotopic and non-retinotopic rotational apparent motion from the earliest evoked peak (around 120ms) and throughout the rest of the visual processing, but only minor correlates of the motion of the reference system itself (mainly around 100-120ms). We suggest that the visual system efficiently discounts the motion of the reference system from early on, allowing a largely reference system independent encoding of the motion of object parts.

Putting low-level vision into global context: Why vision cannot be reduced to basic circuits

To cope with the complexity of vision, most models in neuroscience and computer vision are of hierarchical and feedforward nature. Low-level vision, such as edge and motion detection, is explained by basic low-level neural circuits, whose outputs serve as building blocks for more complex circuits computing higher level features such as shape and entire objects. There is an isomorphism between states of the outer world, neural circuits, and perception, inspired by the positivistic philosophy of the mind. Here, we show that although such an approach is conceptually and mathematically appealing, it fails to explain many phenomena including crowding, visual masking, and non-retinotopic processing.

EEG and fMRI correlates of non-retinotopic motion processing in the human visual system

EEG and fMRI correlates of non-retinotopic motion processing in the human visual system

Infants' visual system nonretinotopically integrates color signals along a motion trajectory.

Whereas early visual processing has been considered primarily retinotopic, recent studies have revealed significant contributions of nonretinotopic processing to the human perception of fundamental visual features. For adult vision, it has been shown that information about color, shape, and size is nonretinotipically integrated along the motion trajectory, which could bring about clear and unblurred perception of a moving object. Since this nonretinotopic processing presumably includes tight and elaborated cooperation among functional cortical modules for different visual attributes, how this processing matures in the course of brain development is an important unexplored question. Here we show that the nonretinotopic integration of color signals is fully developed in infants at five months of age. Using preferential looking, we found significantly better temporal segregation of colors for moving patterns than for flickering patterns, even when the retinal color alternation rate was the same. This effect could be ascribed to the integration of color signals along a motion trajectory. Furthermore, the infants' color segmentation performance was comparable to that of human adults. Given that both the motion processing and color vision of 5-month-old infants are still under development, our findings suggest that nonretinotopic color processing develops concurrently with basic color and motion processing. Our findings not only support the notion of an early presence of cross-modal interactions in the brain, but also indicate the early development of a purposive cross-module interaction for elegant visual computation.

Nonretinotopic visual processing in the brain.

A basic principle in visual neuroscience is the retinotopic organization of neural receptive fields. Here, we review behavioral, neurophysiological, and neuroimaging evidence for nonretinotopic processing of visual stimuli. A number of behavioral studies have shown perception depending on object or external-space coordinate systems, in addition to retinal coordinates. Both single-cell neurophysiology and neuroimaging have provided evidence for the modulation of neural firing by gaze position and processing of visual information based on craniotopic or spatiotopic coordinates. Transient remapping of the spatial and temporal properties of neurons contingent on saccadic eye movements has been demonstrated in visual cortex, as well as frontal and parietal areas involved in saliency/priority maps, and is a good candidate to mediate some of the spatial invariance demonstrated by perception. Recent studies suggest that spatiotopic selectivity depends on a low spatial resolution system of maps that operates over a longer time frame than retinotopic processing and is strongly modulated by high-level cognitive factors such as attention. The interaction of an initial and rapid retinotopic processing stage, tied to new fixations, and a longer lasting but less precise nonretinotopic level of visual representation could underlie the perception of both a detailed and a stable visual world across saccadic eye movements.

The neural correlates of non-retinotopic processing in human visual cortex: a 7T fMRI study

Under normal viewing conditions, due to e.g. the motion of objects, the retinotopic representation of the environment constantly changes. Yet we perceive the world as stable and we easily keep track of moving objects, indicating the presence of non-retinotopic representations in the brain. We investigated the neural correlates of such non-retinotopic processing in the human visual cortex by means of high-resolution fMRI. To this end, we used a Ternus-Pikler display: Three horizontally aligned dark/light gray checker-boards moved back and fourth in a left-right apparent motion while participants kept their gaze fixated above the stimulus, resulting in a non-retinotopic integration of the elements across frames. In a second condition, only two checker-boards were presented at the same location in each frame, thus causing the elements to be integrated retinotopically. Crucially, the checker-boards could be either flickering, i.e. inverting polarity with each frame, or non-flickering. This retinotopic flickering or non-flickering is perceived in the 2-element conditions. In the 3-element conditions, however, because of the non-retinotopic correspondence, the percept is inverted such that the non-flickering checker-boards are perceived to flicker and vice versa. In V1, we found a higher BOLD response for retinotopically flickering stimuli, as expected, but the activity did not depend on the percept of flickering vs. non-flickering. In hMT+ we found a stronger response for the 3-element conditions, which could be due to the apparent motion as well as the larger size of the stimuli. More interestingly, there was also an interaction effect, namely higher activity for perceived flickering than perceived non-flickering. Thus, apart from the retinotopic properties of the stimuli, also the endogenous percept of non-retinotopic origin is reflected in the hMT+ activity.

Non-retinotopic processing in humans vs. retinotopic processing in pigeons.

Non-retinotopic processing in humans vs. retinotopic processing in pigeons.

Corrections to: "Non-retinotopic processing" in Ternus motion displays modeled by spatiotemporal filters

Corrections to: "Non-retinotopic processing" in Ternus motion displays modeled by spatiotemporal filters

"Non-retinotopic processing" in Ternus motion displays modeled by spatiotemporal filters

Recently, M. Boi, H. Ogmen, J. Krummenacher, T. U. Otto, & M. H. Herzog (2009) reported a fascinating visual effect, where the direction of apparent motion was disambiguated by cues along the path of apparent motion, the Ternus-Pikler group motion, even though no actual movement occurs in this stimulus. They referred to their study as a "litmus test" to distinguish "non-retinotopic" (motion-based) from "retinotopic" (retina-based) image processing. We adapted the test to one with simple grating stimuli that could be more readily modeled and replicated their psychophysical results quantitatively with this stimulus. We then modeled our experiments in 3D (x, y, t) Fourier space and demonstrated that the observed perceptual effects are readily accounted for by integration of information within a detector that is oriented in space and time, in a similar way to previous explanations of other motion illusions. This demonstration brings the study of Boi et al. into the more general context of perception of moving objects.

Nonretinotopic Exogenous Attention

Attention is crucial for visual perception because it allows the visual system to effectively use its limited resources by selecting behaviorally and cognitively relevant stimuli from the large amount of information impinging on the eyes. Reflexive, stimulus-driven attention is essential for successful interactions with the environment because it can, for example, speed up responses to life-threatening events. It is commonly believed that exogenous attention operates in the retinotopic coordinates of the early visual system. Here, using a novel experimental paradigm [1], we show that a nonretinotopic cue improves both accuracy and reaction times in a visual search task. Furthermore, the influence of the cue is limited both in space and time, a characteristic typical of exogenous cueing. These and other recent findings show that many more aspects of vision are processed nonretinotopically than previously thought.

Nonretinotopic processing is related to postdictive size modulation in apparent motion

The present study briefly examined how the perceived size of a leading flash would be modulated by trailing motion signals. Observers were presented with two vertical green bars that were followed by white bars with different lengths, which were presented at different locations from the green bars. The task of observers was to discriminate which of the green bars was shorter than the other (Experiment 1) or whether the lengths of the green bars were equal or not (Experiments 2 and 3). One green bar producing apparent motion with the following shorter white bar was reported to be shorter than the other green bar producing apparent motion with the following longer white bar, not only when motion correspondence was determined on the basis of retinal proximity (Experiments 1 and 2),but also when motion correspondence was determined on the basis of nonretinotopic information-that is, a relative location within each perceptual group of bars (Experiment 3). These results indicate that motion processing involving object updating or motion deblurring in the nonretinotopic frame of reference is related to postdictive size modulation.

Motion and tilt aftereffects occur largely in retinal, not in object, coordinates in the Ternus-Pikler display

Recent studies have shown that a variety of aftereffects occurs in a non-retinotopic frame of reference. These findings have been taken as strong evidence that remapping of visual information occurs in a hierarchic manner in the human cortex with an increasing magnitude from early to higher levels. Other studies, however, failed to find non-retinotopic aftereffects. These experiments all relied on paradigms involving eye movements. Recently, we have developed a new paradigm, based on the Ternus-Pikler display, which tests retinotopic vs. non-retinotopic processing without the involvement of eye movements. Using this paradigm, we found strong evidence that attention, form, and motion processing can occur in a non-retinotopic frame of reference. Here, we show that motion and tilt aftereffects are largely retinotopic.

Visual Perception in the Far East: Wolfgang Metzger and Richard Gregory Revisited

A variety of aftereffects was found to be processed non-retinotopically. Other studies failed to find non-retinotopic aftereffects. These experiments relied on paradigms involving eye movements. We have developed a paradigm, based on the Ternus–Pikler display, which tests retinotopic vs non-retinotopic processing without the involvement of eye movements. Here, we presented three disks for about 5 s. The central disk contained a tilted Gabor patch to which observers adapted (the outer disks were grey). After an ISI of 146 ms, the disks were shifted to the right creating the impression of group motion and establishing a non-retinotopic frame of reference. The center disk in the first frame now overlapped with the left most disk. When a test Gabor was presented on this left disk, a strong tilt aftereffect was found, but not when the test Gabor was presented on the center disk. Similar results were found for motion stimuli. Interestingly, invisible retinotopic motion can create a strong aftereffect even though non-retinotopic motion was perceived during adaptation masking the retinotopic motion. Hence, tilt and motion adaptation are processed retinotopically whereas form, motion, and attention were found to be non-retinotopically processed with the Ternus–Pikler paradigm.

Non-retinotopic feature integration decreases response-locked brain activity as revealed by electrical neuroimaging

When presented with dynamic scenes, the brain integrates visual elements across space and time. Such non-retinotopic processing has been intensively studied from a psychophysical point of view, but little is known about the underlying neural processes. Here we used high-density EEG to reveal neural correlates of non-retinotopic feature integration. In an offset-discrimination task we presented sequences of lines for which feature integration depended on a small, endogenous shift of attention. Attention effects were observed in the stimulus-locked evoked potentials but non-retinotopic feature integration was reflected in voltage topographies time-locked to the behavioral response, lasting for about 400 ms. Statistical parametric mapping of estimated current densities revealed that this integration reduced electrical activity in an extensive network of brain areas, with the effects progressing from high-level visual, via frontal, to central ones. The results suggest that endogenously timed neural processes, rather than bottom-up ones, underlie non-retinotopic feature integration.