Non-retinotopic Reference Frame Research Articles

Spinning objects and partial occlusion: Smart neural responses to symmetry

In humans, extrastriate visual areas are strongly activated by symmetry. However, perfect symmetry is rare in natural visual images. Recent findings showed that when parts of a symmetric shape are presented at different points in time the process relies on a perceptual memory buffer. Does this temporal integration need a retinotopic reference frame? For the first time we tested integration of parts both in the temporal and spatial domain, using a non-retinotopic frame of reference. In Experiment 1, an irregular polygonal shape (either symmetric or asymmetric) was partly occluded by a rectangle for 500 ms (T1). The rectangle moved to the opposite side to reveal the other half of the shape, whilst occluding the previously visible half (T2). The reference frame for the object was static: the two parts stimulated retinotopically corresponding receptive fields (revealed over time). A symmetry-specific ERP response from ~300 ms after T2 was observed. In Experiment 2 dynamic occlusion was combined with an additional step at T2: the new half-shape and occluder were rotated by 90°. Therefore, there was a moving frame of reference and the retinal correspondence between the two parts was disrupted. A weaker but significant symmetry-specific response was recorded. This result extends previous findings: global symmetry representation can be achieved in extrastriate areas non-retinotopically, through integration in both temporal and spatial domain.

Non-retinotopic adaptive center-surround modulation in motion processing

The early visual system is organized retinotopically. However, under ecological viewing conditions, motion perception occurs in non-retinotopic coordinates. Even though many studies revealed the central role of non-retinotopic processes, very little is known about their mechanisms and neural correlates. Tadin and colleagues found that increasing the spatial size of a high-contrast drifting-Gabor deteriorates motion-direction discrimination, whereas the opposite occurs with a low-contrast stimulus. The results were proposed to reflect an adaptive center-surround antagonism, whereby at low-contrast the excitatory center dominates whereas at high-contrast suppressive-surround mechanisms become more effective. Because ecological vision is non-retinotopic, we tested the hypothesis that the non-retinotopic system also processes motion information by means of an adaptive center-surround mechanism. We used the Ternus-Pikler display designed to provide either a retinotopic or a non-retinotopic reference-frame. Our results suggest that the non-retinotopic processes underlying motion perception are also mediated by an adaptive center-surround mechanism.

A computational model for reference-frame synthesis with applications to motion perception

As discovered by the Gestaltists, in particular by Duncker, we often perceive motion to be within a non-retinotopic reference frame. For example, the motion of a reflector on a bicycle appears to be circular, whereas, it traces out a cycloidal path with respect to external world coordinates. The reflector motion appears to be circular because the human brain subtracts the horizontal motion of the bicycle from the reflector motion. The bicycle serves as a reference frame for the reflector motion. Here, we present a general mathematical framework, based on vector fields, to explain non-retinotopic motion processing. Using four types of non-retinotopic motion paradigms, we show how the theory works in detail. For example, we show how non-retinotopic motion in the Ternus–Pikler display can be computed.

Spatial properties of non-retinotopic reference frames in human vision

Many visual attributes of a target stimulus are computed according to dynamic, non-retinotopic reference frames. For example, the motion trajectory of a reflector on a bicycle wheel is perceived as orbital, even though it is in fact cycloidal in retinal, as well as spatial coordinates. We cannot perceive the cycloidal motion because the linear motion of the bike is discounted for. In other words, the linear motion common to all bicycle components serves as a non-retinotopic reference frame, with respect to which the residual (orbital) motion of the reflector is computed. Very little is known about the underlying mechanisms involved in formation and operation of non-retinotopic reference frames. Here, we investigate spatial properties of non-retinotopic reference frames. We show that reference frames are not restricted within the boundaries of moving stimuli but extend over space. By using a variation of the Ternus–Pikler paradigm, we show that the spatial extent of a non-retinotopic reference frame is independent of the size of the inducing elements and the target position near the object boundary. While dynamic reference-frames interact with each other significantly, a static reference-frame has no effect on a dynamic one. The magnitude of interactions between two neighboring dynamic reference-frames increases as the distance between them reduces. Finally, our results indicate that the reference-frame strength is significantly attenuated if the locus of attention is shifted to the elements of the neighboring reference instead of the main reference. We suggest that these results can be conceptualized as reference frames that act and interact as fields.

Retinotopy of visual masking and non-retinotopic perception during masking.

Due to the movements of the observer and those of objects in the environment, retinotopic representations are highly unstable during ecological viewing conditions. The phenomenal stability of our perception suggests that retinotopic representations are transformed into non-retinotopic representations. It remains to show, however, which visual processes operate under retinotopic representations and which ones operate under non-retinotopic representations. Visual masking refers to the reduced visibility of one stimulus, called the target, due to the presence of a second stimulus, called the mask. Masking has been used extensively to study the dynamic aspects of visual perception. Previous studies using Saccadic Stimulus Presentation Paradigm (SSPP) suggested both retinotopic and non-retinotopic bases for visual masking. In order to understand how the visual system deals with retinotopic changes induced by moving targets, we investigated the retinotopy of visual masking and the fate of masked targets under conditions that do not involve eye movements. We have developed a series of experiments based on a radial Ternus-Pikler display. In this paradigm, the perceived Ternus-Pikler motion is used as a non-retinotopic reference frame to pit retinotopic against non-retinotopic visual masking hypothesis. Our results indicate that both metacontrast and structure masking are retinotopic. We also show that, under conditions that allow observers to read-out effectively non-retinotopic feature attribution, the target becomes visible at a destination different from its retinotopic/spatiotopic location. We discuss the implications of our findings within the context of ecological vision and dynamic form perception.

How are non-retinotopic motion signals integrated? -A high-density EEG study

Objects moving in the visual scene cause retinal displacements that are not the result of motor commands and thus cannot be accounted for by efference copies. Yet, we easily keep track of moving objects even without following them with our gaze. Here, we investigated the neural correlates of non-retinotopic motion integration using high-density EEG. We presented three disks that either flickered at the same location (retinotopic reference frame) or moved left-right in apparent motion, creating a non-retinotopic reference frame in which the features of the disks are integrated across retinal positions. In one disk, a notch was either changing positions across frames in a rotating fashion, or stayed in the same position. The notch then started or stopped rotating after a random number of frames. We found stronger EEG responses for rotating than for static notches. In the novel state (first frame of rotating or static), this effect occurs in the N2 peak and resembles a motion-onset detection signal. Inverse solutions point to the right middle temporal gyrus as the underlying source. Importantly, these results hold for both the retinotopic and the non-retinotopic reference frames, indicating that the rotation encoding is independent of reference frame.

Neural correlates of non-retinotopic motion integration

Under normal viewing conditions, due to the motion of objects and to eye movements, the retinotopic representation of the environment constantly changes. Yet we perceive the world as stable, and we easily keep track of moving objects. Here, we investigated the neural correlates of non-retinotopic motion integration using high-density EEG. We used a Ternus-Pikler display to establish either a retinotopic or non-retinotopic frame of reference. Three disks were presented for 250 ms followed by an ISI of 150 ms. The disks then reappeared either at the same location (retinotopic reference frame), or shifted sideways (non-retinotopic reference frame). After another ISI, the sequence started over again. In the middle disk, a dot was either changing positions across frames in a rotating fashion, or stayed in the same position. Every 5th to 9th frame, the dot started or stopped rotating, and observers reported this with a button-press. We found higher EEG responses for rotating than static dots. This effect occurred rather late (>200 ms), i.e. after basic stimulus encoding (P1 component). Importantly, these results hold for both the retinotopic and the non-retinotopic conditions, indicating that the encoding of rotation does not depend on reference frame. In line with this, reference frame effects were observed at earlier latencies and did not interact with rotation effects. Electrical source imaging showed that the underlying neural processing of this non-retinotopic effect seems to be located partially in extrastriate visual areas.

Nonretinotopic Exogenous Attention

Attention is crucial for visual perception because it allows the visual system to effectively use its limited resources by selecting behaviorally and cognitively relevant stimuli from the large amount of information impinging on the eyes. Reflexive, stimulus-driven attention is essential for successful interactions with the environment because it can, for example, speed up responses to life-threatening events. It is commonly believed that exogenous attention operates in the retinotopic coordinates of the early visual system. Here, using a novel experimental paradigm [1], we show that a nonretinotopic cue improves both accuracy and reaction times in a visual search task. Furthermore, the influence of the cue is limited both in space and time, a characteristic typical of exogenous cueing. These and other recent findings show that many more aspects of vision are processed nonretinotopically than previously thought.

Nonretinotopic processing is related to postdictive size modulation in apparent motion

The present study briefly examined how the perceived size of a leading flash would be modulated by trailing motion signals. Observers were presented with two vertical green bars that were followed by white bars with different lengths, which were presented at different locations from the green bars. The task of observers was to discriminate which of the green bars was shorter than the other (Experiment 1) or whether the lengths of the green bars were equal or not (Experiments 2 and 3). One green bar producing apparent motion with the following shorter white bar was reported to be shorter than the other green bar producing apparent motion with the following longer white bar, not only when motion correspondence was determined on the basis of retinal proximity (Experiments 1 and 2),but also when motion correspondence was determined on the basis of nonretinotopic information-that is, a relative location within each perceptual group of bars (Experiment 3). These results indicate that motion processing involving object updating or motion deblurring in the nonretinotopic frame of reference is related to postdictive size modulation.

Motion and tilt aftereffects occur largely in retinal, not in object, coordinates in the Ternus-Pikler display

Recent studies have shown that a variety of aftereffects occurs in a non-retinotopic frame of reference. These findings have been taken as strong evidence that remapping of visual information occurs in a hierarchic manner in the human cortex with an increasing magnitude from early to higher levels. Other studies, however, failed to find non-retinotopic aftereffects. These experiments all relied on paradigms involving eye movements. Recently, we have developed a new paradigm, based on the Ternus-Pikler display, which tests retinotopic vs. non-retinotopic processing without the involvement of eye movements. Using this paradigm, we found strong evidence that attention, form, and motion processing can occur in a non-retinotopic frame of reference. Here, we show that motion and tilt aftereffects are largely retinotopic.

Visual Perception in the Far East: Wolfgang Metzger and Richard Gregory Revisited

A variety of aftereffects was found to be processed non-retinotopically. Other studies failed to find non-retinotopic aftereffects. These experiments relied on paradigms involving eye movements. We have developed a paradigm, based on the Ternus–Pikler display, which tests retinotopic vs non-retinotopic processing without the involvement of eye movements. Here, we presented three disks for about 5 s. The central disk contained a tilted Gabor patch to which observers adapted (the outer disks were grey). After an ISI of 146 ms, the disks were shifted to the right creating the impression of group motion and establishing a non-retinotopic frame of reference. The center disk in the first frame now overlapped with the left most disk. When a test Gabor was presented on this left disk, a strong tilt aftereffect was found, but not when the test Gabor was presented on the center disk. Similar results were found for motion stimuli. Interestingly, invisible retinotopic motion can create a strong aftereffect even though non-retinotopic motion was perceived during adaptation masking the retinotopic motion. Hence, tilt and motion adaptation are processed retinotopically whereas form, motion, and attention were found to be non-retinotopically processed with the Ternus–Pikler paradigm.

Perceptual Learning in a Nonretinotopic Frame of Reference

Perceptual learning is the ability to improve perception through practice. Perceptual learning is usually specific for the task and features learned. For example, improvements in performance for a certain stimulus do not transfer if the stimulus is rotated by 90° or is presented at a different location. These findings are usually taken as evidence that orientation-specific, retinotopic encoding processes are changed during training. In this study, we used a novel masking paradigm in which the offset in an invisible, oblique vernier stimulus was perceived in an aligned vertical or horizontal flanking stimulus presented at a different location. Our results show that learning is specific for the perceived orientation of the vernier offset but not for its actual orientation and location. Specific encoding processes cannot be invoked to explain this improvement. We propose that perceptual learning involves changes in nonretinotopic, attentional readout processes.

A (fascinating) litmus test for human retino- vs.non-retinotopic processing

In human vision, the optics of the eye map neighboring points of the environment onto neighboring photoreceptors in the retina. This retinotopic encoding principle is preserved in the early visual areas. Under normal viewing conditions, due to the motion of objects and to eye movements, the retinotopic representation of the environment undergoes fast and drastic shifts. Yet, perceptually our environment appears stable suggesting the existence of non-retinotopic representations in addition to the well-known retinotopic ones. Here, we present a simple psychophysical test to determine whether a given visual process is accomplished in retino- or non-retinotopic coordinates. As examples, we show that visual search and motion perception can occur within a non-retinotopic frame of reference. These findings suggest that more mechanisms than previously thought operate non-retinotopically. Whether this is true for a given visual process can easily be found out with our "litmus test."