Scene Layout Research Articles

Perception of Scene Layout from Optical Contact, Shadows, and Motion

Kersten et al (1997 Perception 26 171-192) found that the perceived motion of an object in a 3-D scene was determined by the motion of a shadow. In the present study, we compared the effect of a shadow to that of a second object on the ground in determining the perceived position in depth of a floating object in both dynamic and stationary scenes. Changing the second (lower) object from textured to dark increased the influence of the second object on the judged position of the first object. Giving the second object zero thickness had this effect only if it was also dark. Variations in the height of the floating object were important with a second object but not with a shadow, in motion scenes. With alternative shadows present, the position of the floating object was determined primarily by matching speeds, with matching sizes as a secondary factor. These results show some similarities but important differences between the effect of a second object and that of a shadow.

Interaction of optical contact, shadows and motion in determining perceived scene layout

Interaction of optical contact, shadows and motion in determining perceived scene layout

The time course with which representations of scene layout become functional

The time course with which representations of scene layout become functional

Independent effects of object size and location on scene layout extrapolation.

Independent effects of object size and location on scene layout extrapolation.

Vantage point in traumatic memory.

Persons with posttraumatic stress disorder (PTSD) spontaneously recalled the traumatic event that led to their condition from either a field or an observer vantage point. In the former case, participants recollected the event as if they were seeing it again through their own eyes; in the latter case, the event was recalled from the perspective of a detached spectator. Analysis of the two types of recollections revealed marked differences in their contents. Whereas field memories afforded richer accounts of the affective reactions, somatic sensations, and psychological states that participants experienced during the focal trauma, observer memories contained more information about participants' physical appearance and actions and the spatial layout of the traumatic scene. Observer trauma memories were also experienced as less emotional and anxiety provoking than field trauma memories. The discussion focuses on the clinical implications of these findings and prospects for future research on traumatic memory.

Implicit scene learning is viewpoint dependent.

When novel scenes are encoded, the representations of scene layout are generally viewpoint specific. Past studies of scene recognition have typically required subjects to explicitly study and encode novel scenes, but in everyday visual experience, it is possible that much scene learning occurs incidentally. Here, we examine whether implicitly encoded scene layouts are also viewpoint dependent. We used the contextual cuing paradigm, in which search for a target is facilitated by implicitly learned associations between target locations and novel spatial contexts (Chun & Jiang, 1998). This task was extended to naturalistic search arrays with apparent depth. To test viewpoint dependence, the viewpoint of the scenes was varied from training to testing. Contextual cuing and, hence, scene context learning decreased as the angular rotation from training viewpoint increased. This finding suggests that implicitly acquired representations of scene layout are viewpoint dependent.

Vertical disparity can alter perceived direction.

It has been well established that vertical disparity is involved in perception of the three-dimensional layout of a visual scene. The goal of this paper was to examine whether vertical disparities can alter perceived direction. We dissociated the common relationship between vertical disparity and the stimulus direction by applying a vertical magnification to the image presented to one eye. We used a staircase paradigm to measure whether perceived straight-ahead depended on the amount of vertical magnification in the stimulus. Subjects judged whether a test dot was flashed to either the left or the right side of straight-ahead. We found that perceived straight-ahead did indeed depend on the amount of vertical magnification but only after subjects adapted (for 5 min) to vertical scale (and only in five out of nine subjects). We argue that vertical disparity is a factor in the calibration of the relationship between eye-position signals and perceived direction.

Subjective contours at line terminations depend on scene layout analysis, not image processing.

Subjective contours at abutting lines, commonly attributed to levels of the visual system where image properties are directly encoded, are shown to require an explanation at the level of visual surface understanding. By the use of stereoscopic displays, subjective contours could either be abolished where they were monocularly strong or be created where they were monocularly nonexistent. Furthermore, it was shown that subjective contours are determined by the boundaries of occluded rather than occluding regions. The results cannot be accounted for by adding a stereo component to existing image-based theories, such as those based on end-stopped cells. Instead, any plausible process must explicitly determine the position of bounding surface contours in the scene. Minimum requirements of such a process are given. Subjective contours are of two major varieties: those that complete existing luminance contours across gaps (Schumann, 1900; Kanizsa, 1974; see Figure 1, top) and those that form along a set of line terminators (Ehrenstein, 1941/1987), especially along two sets of abutting line terminators (Kanizsa, 1974; see Figure 1, bottom left). In both cases, it is generally agreed that subjective contours serve the function of delineating the boundaries of occluding surfaces when luminance contours are camouflaged in the image by the luminance of the surrounding field. With respect to mechanism, there is considerable disagreement. Several physiological findings of great interest, discussed in detail, subsequently have encouraged the continuing widespread view that subjective contours are the product of low-level physiological mechanisms acting at early levels of the visual system that directly encode image properties. This marries function and mechanism in an unusually persuasive and satisfying manner. It is the goal of this article to present a different view based on striking new findings. We used stereograms to manipulate the three-dimensional (3-D) placement of the contours of the abutting line effect. This is the simplest configuration giving rise to subjective contours and the one most readily explained by low-level theories (Treisman, Cavanagh, Fischer, Ramachandran, & von der Heydt, 1990). On the basis of our results, we argue that such subjective contours can be explained only at the level of scene analysis. We sketch out some of the complexities an adequate explanation must encompass. We also attempt to analyze in some detail why current image-based mechanisms cannot provide such an explanation.

Subjective contours at line terminations depend on scene layout analysis, not image processing.

Subjective contours at abutting lines, commonly attributed to levels of the visual system where image properties are directly encoded, are shown to require an explanation at the level of visual surface understanding. By the use of stereoscopic displays, subjective contours could either be abolished where they were monocularly strong or be created where they were monocularly nonexistent. Furthermore, it was shown that subjective contours are determined by the boundaries of occluded rather than occluding regions. The results cannot be accounted for by adding a stereo component to existing image-based theories, such as those based on end-stopped cells. Instead, any plausible process must explicitly determine the position of bounding surface contours in the scene. Minimum requirements of such a process are given. Subjective contours are of two major varieties: those that complete existing luminance contours across gaps (Schumann, 1900; Kanizsa, 1974; see Figure 1, top) and those that form along a set of line terminators (Ehrenstein, 1941/1987), especially along two sets of abutting line terminators (Kanizsa, 1974; see Figure 1, bottom left). In both cases, it is generally agreed that subjective contours serve the function of delineating the boundaries of occluding surfaces when luminance contours are camouflaged in the image by the luminance of the surrounding field. With respect to mechanism, there is considerable disagreement. Several physiological findings of great interest, discussed in detail, subsequently have encouraged the continuing widespread view that subjective contours are the product of low-level physiological mechanisms acting at early levels of the visual system that directly encode image properties. This marries function and mechanism in an unusually persuasive and satisfying manner. It is the goal of this article to present a different view based on striking new findings. We used stereograms to manipulate the three-dimensional (3-D) placement of the contours of the abutting line effect. This is the simplest configuration giving rise to subjective contours and the one most readily explained by low-level theories (Treisman, Cavanagh, Fischer, Ramachandran, & von der Heydt, 1990). On the basis of our results, we argue that such subjective contours can be explained only at the level of scene analysis. We sketch out some of the complexities an adequate explanation must encompass. We also attempt to analyze in some detail why current image-based mechanisms cannot provide such an explanation.

Designing for the Task: Sometimes 2-D is Just Plane Better

There is a great military command and control need for better depiction of 3-D environments. Standard 2-D depictions can lead to inattention to altitude, especially for relative novices, and are generally inadequate. In a series of experiments we found that tasks requiring understanding the shapes of objects or terrain layout are much better performed using 3-D views, however tasks requiring precise judgments about the relative positions of objects or terrain locations were much better performed using 2-D views. We propose an interface concept called “Orient and Operate” for designing command and control displays. Users orient to the layout of a scene using a 3-D view, but then switch to 2-D views to interact or operate on it. In related work, we have found that 3-D icons are a mixed blessing: they are poor for identification, yet they code other attributes clearly (e.g. attitude), but this benefit is unrelated to 3-D depiction, per se.

A neural model of motion processing and visual navigation by cortical area MST.

Cells in the dorsal medial superior temporal cortex (MSTd) process optic flow generated by self-motion during visually guided navigation. A neural model shows how interactions between well-known neural mechanisms (log polar cortical magnification, Gaussian motion-sensitive receptive fields, spatial pooling of motion-sensitive signals and subtractive extraretinal eye movement signals) lead to emergent properties that quantitatively simulate neurophysiological data about MSTd cell properties and psychophysical data about human navigation. Model cells match MSTd neuron responses to optic flow stimuli placed in different parts of the visual field, including position invariance, tuning curves, preferred spiral directions, direction reversals, average response curves and preferred locations for stimulus motion centers. The model shows how the preferred motion direction of the most active MSTd cells can explain human judgments of self-motion direction (heading), without using complex heading templates. The model explains when extraretinal eye movement signals are needed for accurate heading perception, and when retinal input is sufficient, and how heading judgments depend on scene layouts and rotation rates.

Dynamical use of different sources of information in heading judgments from retinal flow.

The optic flow arising in the eyes of an observer during self-motion is influenced by the occurrence of eye movements. The determination of heading during eye movements may be based on the pattern of retinal image motion (the retinal flow) or on an additional use of an extraretinal eye-movement signal. Previous research has presented support for either of these hypotheses, depending on the movement geometry and the layout of the visual scene. A special situation in which all previous studies unequivocally have agreed that an extra-retinal signal is required occurs when the visual scene consists of a single frontoparallel plane. In this situation eye movements shift the center of expansion on the retina to a location that does not correspond to the direction of self-movement. Without extraretinal input, human observers confuse the center of expansion with their heading and show a systematical heading estimation error. We reexamined and further investigated this situation. We presented retinal flow stimuli on a large projection screen in the absence of extra-retinal input and varied stimulus size, presentation duration, and orientation of the plane. In contrast to previous studies we found that in the case of a perpendicular approach toward the plane, heading judgments can be accurate. Accurate judgments were observed when the field of view was large (90 degrees x 90 degrees) and the stimulus duration was short (< or = 0.5 s). For a small field of view or a prolonged stimulus presentation, a systematic and previously described error appeared that is related to the radial structure of the flow field and the location of the center of expansion. An oblique approach toward the plane results in an ambiguous flow field with two mathematically possible solutions for heading. In this situation, when the stimulus duration was short, subjects reported a perceived heading midway between these two solutions. For longer flow sequences, subjects again chose the center of expansion. Our results suggest a dynamical change in the analysis or interpretation of retinal flow during heading perception.

The parahippocampal place area: recognition, navigation, or encoding?

The parahippocampal place area (PPA) has been demonstrated to respond more strongly in fMRI to scenes depicting places than to other kinds of visual stimuli. Here, we test several hypotheses about the function of the PPA. We find that PPA activity (1) is not affected by the subjects' familiarity with the place depicted, (2) does not increase when subjects experience a sense of motion through the scene, and (3) is greater when viewing novel versus repeated scenes but not novel versus repeated faces. Thus, we find no evidence that the PPA is involved in matching perceptual information to stored representations in memory, in planning routes, or in monitoring locomotion through the local or distal environment but some evidence that it is involved in encoding new perceptual information about the appearance and layout of scenes.

Wide-angle memories of close-up scenes: a demonstration of boundary extension.

Boundary extension refers to a tendency to remember seeing a greater expanse of a scene than was shown in a photograph. It is hypothesized that the view shown in the stimulus activates expectations about the scene's layout just outside the picture's borders. Following presentation, the viewer remembers having seen this expected information, and this yields boundary extension. We provide photographs and instructions for conducting two brief demonstrations of the phenomenon and provide materials for a related class experiment on the journal's World-Wide Web site. These demonstrations of boundary extension provide graphic illustrations of the role of schematic expectancies in the representation of scenes and help to illustrate the role of real-world knowledge in cognition.

Emulating the Visual Receptive-Field Properties of MST Neurons with a Template Model of Heading Estimation

We have proposed previously a computational neural-network model by which the complex patterns of retinal image motion generated during locomotion (optic flow) can be processed by specialized detectors acting as templates for specific instances of self-motion. The detectors in this template model respond to global optic flow by sampling image motion over a large portion of the visual field through networks of local motion sensors with properties similar to those of neurons found in the middle temporal (MT) area of primate extrastriate visual cortex. These detectors, arranged within cortical-like maps, were designed to extract self-translation (heading) and self-rotation, as well as the scene layout (relative distances) ahead of a moving observer. We then postulated that heading from optic flow is directly encoded by individual neurons acting as heading detectors within the medial superior temporal (MST) area. Others have questioned whether individual MST neurons can perform this function because some of their receptive-field properties seem inconsistent with this role. To resolve this issue, we systematically compared MST responses with those of detectors from two different configurations of the model under matched stimulus conditions. We found that the characteristic physiological properties of MST neurons can be explained by the template model. We conclude that MST neurons are well suited to support self-motion estimation via a direct encoding of heading and that the template model provides an explicit set of testable hypotheses that can guide future exploration of MST and adjacent areas within the superior temporal sulcus.

Priming Spatial Layout of Scenes

Observers responded to full-color images of scenes by indicating which of two critical objects was closer in the pictorial space. These target images were preceded by prime images of the same scene sans critical objects, or by control primes or different-scene primes. Reaction times were faster following same-scene primes than following the various control and different-scene primes. Same-scene facilitation was obtained with color primes, line-drawing primes, and primes with shifted views. The effect occurred with natural scenes having gist and simple artificial scenes having little or no gist. The results indicate that prime-induced representations influence the perception of spatial layout in pictures.

Moving cast shadows induce apparent motion in depth.

Phenomenally strong visual illusions are described in which the motion of an object's cast shadow determines the perceived 3-D trajectory of the object. Simply adjusting the motion of a shadow is sufficient to induce dramatically different apparent trajectories of the object casting the shadow. Psychophysical results obtained with the use of 3-D graphics are reported which show that: (i) the information provided by the motion of an object's shadow overrides other strong sources of information and perceptual biases, such as the assumption of constant object size and a general viewpoint; (ii) the natural constraint of shadow darkness plays a role in the interpretation of a moving image patch as a shadow, but under some conditions even unnatural light shadows can induce apparent motion in depth of an object; (iii) when shadow motion is caused by a moving light source, the visual system incorrectly interprets the shadow motion as consistent with a moving object, rather than a moving light source. The results support the hypothesis that the human visual system incorporates a stationary light-source constraint in the perceptual processing of spatial layout of scenes.

Neural responses to velocity gradients in macaque cortical area MT.

Visual motion, i.e. the pattern of changes on the retinae caused by the motion of objects or the observer through the environment, contains important cues for the accurate perception of the three-dimensional layout of the visual scene. In this study, we investigate if neurons in the visual system, specifically in area MT of the macaque monkey, are able to differentiate between various velocity gradients. Our stimuli were random dot patterns designed to eliminate stimulus variables other than the orientation of a velocity gradient. We develop a stimulus space ("deformation space") that allows us to easily parameterize our stimuli. We demonstrate that a substantial proportion of MT cells show tuned responses to our various velocity gradients, often exceeding the response evoked by an optimized flat velocity profile. This suggests that MT cells are able to represent complex aspects of the visual environment and that their properties make them well suited as building blocks for the complex receptive field properties encountered in higher areas, such as area MST to which many cells in area MT project.

Perceiving scene layout through an aperture during visually simulated self-motion.

Perceiving scene layout through an aperture during visually simulated self-motion.

Perceiving scene layout through an aperture during visually simulated self-motion.

This study tested scene perception during depicted self-movement. In Experiment 1, viewers reproduced angular scene configurations, revealed over time by camera motions in depth. Viewers used visible flow to update off-screen locations of landmarks that had been drawn from view and thereby perceived scenes in depth beyond the display's boundaries. Experiments 2 and 3 tested whether offscreen space is perceived as a function of depicted velocity and time and whether information from prior views is used. Viewers predicted when lateral movement would reveal a peripheral landmark whose location was shown in a prior panoramic view. Accurate responses, observed under certain conditions, would result if viewers perceived space according to S = integral of V dt (D. Algom & L. Cohen-Raz, 1984, 1987; R. Jagacinski, W. Johnson, & R. Miller, 1983). When required to picture wide spans in space yet to emerge, however, viewers responded as if these spans were compressed. The theoretical implications of the ability to retrieve, transform, and apply information after a discontinuous transition from a prior view are discussed.