3D Shape Perception Research Articles

Monocular and binocular perception of 3D shape: The role of a priori constraints

Monocular and binocular perception of 3D shape: The role of a priori constraints

Adaptation to the relation between visual cues affects perception of 3D shape

Adaptation to the relation between visual cues affects perception of 3D shape

The perception of 3D shape from anisotropic texture patterns

The perception of 3D shape from anisotropic texture patterns

The perception of 3D shape from surface contours

The perception of 3D shape from surface contours

Effects of visible contact point on visual–haptic integration in 3D shape perception

Effects of visible contact point on visual–haptic integration in 3D shape perception

Release from cross-orientation suppression facilitates 3D shape perception.

Cross-orientation suppression (COS) in striate cortex has been implicated in the efficient encoding of visual stimuli. We show that release from COS facilitates the decoding of 3-D shape. In planar surfaces overlaid with textures, slanting the surface can increase the visibility of the component parallel to the slant. Since this component provides the orientation flows that signify 3-D shape, the enhancement of visibility facilitates 3-D slant perception. Contrast thresholds reveal that this enhancement results from a decrease in COS when 3-D slant creates a frequency mismatch between texture components. We show that coupling compressive nonlinearities in LGN neurons with expansive nonlinearities in cortical neurons can model the frequency-specific component of suppression.

A tactile stimulator for studying passive shape perception

We describe a computer-controlled tactile stimulator for use in human psychophysical and monkey neurophysiological studies of 3D shape perception. The stimulator is constructed primarily of commercially available parts, as well as a few custom-built pieces for which we will supply diagrams upon request. There are two components to the stimulator: a tactile component and a hand positioner component. The tactile component consists of multiple stimulating units that move about in a Cartesian plane above the restrained hand. Each stimulating unit contains a servo-controlled linear motor with an attached small rotary stepper motor, allowing arbitrary stimulus shapes to contact the skin through vibration, static indentation, or scanning. The hand positioner component modifies the conformation of the restrained hand through a set of mechanical linkages under motorized control. The present design controls the amount of spread between digits 2 and 3, the spread between digits 4 and 3, and the degree to which digit 3 is flexed or extended, thereby simulating different conformations of the hand in contact with objects. This design is easily modified to suit the needs of the experimenter. Because the two components of the stimulator are independently controlled, the stimulator allows for parametric study of the mechanoreceptive and proprioceptive contributions to 3D tactile shape perception.

Perception of 3D shape in context: Contrast and assimilation

Whereas integration of shape and surround is held to occur through cue-dependent representations, we show that both cue-invariant and cue-dependent representations are involved. A central hinged plane and larger flanking plane were defined by either binocular disparity or motion. In a ‘within-cue’ condition, shape and surround were defined by the same cue and in a ‘cross-cue’ condition they were defined by a different cue. Observers compared the dihedral angle of the central shape with a constant reference. When the central shape was defined by disparity, the surround stimuli invoked a contrast bias in the within-cue condition, but shape assimilation occurred in the cross-cue condition. When the central shape was defined by motion there were overall no significant results, but if a contrast bias was observed, it was in the within-cue condition where integration could occur through cue-dependent representations.

Haptic perception disambiguates visual perception of 3D shape

We studied the influence of haptics on visual perception of three-dimensional shape. Observers were shown pictures of an oblate spheroid in two different orientations. A gauge-figure task was used to measure their perception of the global shape. In the first two sessions only vision was used. The results showed that observers made large errors and interpreted the oblate spheroid as a sphere. They also misinterpreted the rotated oblate spheroid for a prolate spheroid. In two subsequent sessions observers were allowed to touch the stimulus while performing the task. The visual input remained unchanged: the observers were looking at the picture and could not see their hands. The results revealed that observers perceived a shape that was different from the vision-only sessions and closer to the veridical shape. Whereas, in general, vision is subject to ambiguities that arise from interpreting the retinal projection, our study shows that haptic input helps to disambiguate and reinterpret the visual input more veridically.

Using curvature information in haptic shape perception of 3D objects

Are humans able to perceive the circularity of a cylinder that is grasped by the hand? This study presents the findings of an experiment in which cylinders with a circular cross-section had to be distinguished from cylinders with an elliptical cross-section. For comparison, the ability to distinguish a square cuboid from a rectangular cuboid was also investigated. Both elliptical and rectangular shapes can be characterized by the aspect ratio, but elliptical shapes also contain curvature information. We found that an elliptical shape with an aspect ratio of only 1.03 could be distinguished from a circular shape both in static and dynamic touch. However, for a rectangular shape, the aspect ratio needed to be about 1.11 for dynamic touch and 1.15 for static touch in order to be discernible from a square shape. We conclude that curvature information can be employed in a reliable and efficient manner in the perception of 3D shapes by touch.

Sensitivity to luminance and chromaticity gradients in a complex scene

Image gradients--smooth changes in color and luminance--may be caused by intrinsic surface reflectance properties or extrinsic illumination phenomena, including shading, shadowing, and inter-reflections. In turn, image gradients may provide the visual system with information concerning the origin of these factors, such as the orientation of surfaces with respect to the light source. Color gradients induced by mutual illumination (MI) may play a similar role to that of luminance gradients in shape-from-shading algorithms; it has been shown that 3D shape perception modulates the influence of MI on surface color perception (M. G. Bloj, D. Kersten, & A. C. Hurlbert, 1999). In this study, we assess human sensitivity to changes in color and luminance gradients that arise from changes in the light source position, within a complex scene. In Experiment 1, we tested whether observers were able to discriminate between gradients due to different light source positions. We found that observers reliably detected a change in the gradient information when the light source position differed by only 4 deg from the reference scene. This sensitivity was mainly based on the luminance information in the gradient (Experiments 2 and 3). Some observers make use of the spatial distribution of chromaticity and luminance values within gradients when discriminating between them (Experiment 4). The high sensitivity to gradient differences supports the notion that gradients contain information that may assist in the recovery of 3D shape and scene configuration properties.

Binocular disparity only comes into play when everything else fails; a finding with broader implications than one might suppose

This paper calls attention to research showing that binocular disparity, which is an effective cue to depth, plays a secondary role, at best, in the perception of 3D shape. This claim has implications both for how shape should be studied and how this unique perceptual property should be modeled. These issues are discussed from a historical perspective, which shows how the failure to appreciate the importance of the Gestalt grouping principle called 'Figure-Ground Organization' led to many unfruitful efforts. It also calls attention to how this situation can be remedied.

The influence of training views on infants' long‐term memory for simple 3D shapes

This investigation explored infants' ability to retrieve a memory for a simple 3D shape from a novel view following a 24-hr delay. Tests of memory for shape in infancy have typically used extremely short delay intervals between familiarization and test in examining the ability to equate between substantially different views of a 3D object. The current study used longer delays to assess the content of a long-term memory representation. Infants 3-4 months of age learned to kick to move a mobile displaying a simple 3D shape (brick or cylinder). Results of three experiments show that infants can recognize 3D shapes in a novel viewpoint across a 24-hr delay, provided that experience with a sufficiently wide range of views is available during training. The results suggest a capacity for the perception of 3D shape that enables access, across long delays, to a memory representation of sufficient detail that discrimination between two simple shapes (i.e., a cylinder and a brick) is possible. The results suggest that this representation is of a sufficiently abstract nature that perception of the 3D form of the object, independent of the changes in specific features accompanying changes in viewpoint, is also possible. This finding suggests that infants, like adults, possess a functional memory system for the distal shape of simple 3D objects, and can transfer training to a novel view using long-term memory, but that this ability is not as strong as in the mature system. These results have implications for the development of shape perception and for theories of object recognition in general.

Neural Correlates of Disparity-Defined Shape Discrimination in the Human Brain

Binocular disparity, the slight differences between the images registered by our two eyes, provides an important cue when estimating the three-dimensional (3D) structure of the complex environment we inhabit. Sensitivity to binocular disparity is evident at multiple levels of the visual hierarchy in the primate brain, from early visual cortex to parietal and temporal areas. However, the relationship between activity in these areas and key perceptual functions that exploit disparity information for 3D shape perception remains an important open question. Here we investigate the link between human cortical activity and the perception of disparity-defined shape, measuring fMRI responses concurrently with psychophysical shape judgments. We parametrically degraded the coherence of shapes by shuffling the spatial position of dots whose disparity defined the 3D structure and investigated the effect of this stimulus manipulation on both cortical activity and shape discrimination. We report significant relationships between shape coherence and fMRI response in both dorsal (V3, hMT+/V5) and ventral (LOC) visual areas that correspond to the observers' discrimination performance. In contrast to previous suggestions of a dichotomy of disparity-related processes in the ventral and dorsal streams, these findings are consistent with proposed interactions between these pathways that may mediate a continuum of processes important in perceiving 3D shape from coarse contour segmentation to fine curvature estimation.

The effects of phase on the perception of 3D shape from texture: Psychophysics and modeling

Two experiments are reported in which observers judged the apparent shapes of elliptical cylinders with eight different textures that were presented with scrambled and unscrambled phase spectra. The results revealed that the apparent depths of these surfaces varied linearly with the ground truth in all conditions, and that the overall magnitude of surface relief was systematically underestimated. In general, the apparent depth of a surface is significantly attenuated when the phase spectrum of its texture is randomly scrambled, though the magnitude of this effect varies for different types of texture. A new computational model of 3D shape from texture is proposed in which apparent depth is estimated from the relative density of edges in different local regions of an image, and the predictions of this model are highly correlated with the observers’ judgments.

Visual saliency and texture segregation without feature gradient.

A central notion in the study of texture segregation is that of feature gradient (or feature contrast). In orientation-based texture segregation, orientation gradients have indeed played a fundamental role in explaining behavioral results. Here, however, we show that general, smoothly varying, orientation-defined textures (ODTs) exhibit striking perceptual singularities that are completely unpredictable from orientation gradients. These singularities defy not only popular texture segregation theories but also virtually all computational segmentation methods, and they confound previous behavioral studies with smoothly varying ODTs. We provide psychophysical evidence that perceptual singularities in smooth ODTs are salient visual features consistent across observers and with significant effect on the perception and segregation of oriented textures. We further show that, although orientation gradients cannot predict them, perceptual singularities in smooth ODTs emerge directly from, and can be spatially localized by, two ODT curvatures. Given the traditional role of feature gradients in early vision, the significance of these findings extends well beyond orientation-based texture segregation to issues ranging from curve integration and fragment grouping, through the perception of 3D shape, to the functional organization of the primary visual cortex.

Sketching shiny surfaces

Many materials including water, plastic, and metal have specular surface characteristics. Specular reflections have commonly been considered a nuisance for the recovery of object shape. However, the way that reflections are distorted across the surface depends crucially on 3D curvature, suggesting that they could, in fact, be a useful source of information. Indeed, observers can have a vivid impression of, 3D shape when an object is perfectly mirrored (i.e., the image contains nothing but specular reflections). This leads to the question what are the underlying mechanisms of our visual system to extract this 3D shape information from a perfectly mirrored object. In this paper we propose a biologically motivated recurrent model for the extraction of visual features relevant for the perception of 3D shape information from images of mirrored objects. We qualitatively and quantitatively analyze the results of computational model simulations and show that bidirectional recurrent information processing leads to better results than pure feedforward processing. Furthermore, we utilize the model output to create a rough nonphotorealistic sketch representation of a mirrored object, which emphasizes image features that are mandatory for 3D shape perception (e.g., occluding contour and regions of high curvature). Moreover, this sketch illustrates that the model generates a representation of object features independent of the surrounding scene reflected in the mirrored object.

Identification of 3D shape from texture and motion across the visual field

Little is known about the perception of 3D shape in the visual periphery. Here we ask whether identification accuracy in shape-from-texture and shape-from-motion tasks can be equated across the visual field with sufficient stimulus magnification. Both tasks employed 3D surfaces comprising hills, valleys, and plains in three possible locations, yielding a 27 alternative forced-choice task (27AFC). Participants performed the task at eccentricities of 0 to 16 deg in the right visual field over a 64-fold range of stimulus sizes. Performance reached ceiling levels at all eccentricities, indicating that stimulus magnification was sufficient to compensate for eccentricity-dependent sensitivity loss. The parameter E(2) (in the equation F = 1 + E / E(2)) was used to characterize the rate at which stimulus size must increase with eccentricity (E) to achieve foveal levels of performance. Three parameter models (mu, sigma, and E(2)) captured most of the variability in the psychometric functions relating stimulus size and eccentricity to accuracy for all participants' data in the two experiments. For the shape-from-texture task, the average E(2) was 1.52, and for the shape-from-motion task, it was 0.61. The E(2) values indicate that sensitivity to structure from motion declines at a faster rate with eccentricity than does sensitivity to structure from texture. Although size scaling with F = 1 + E / E(2) eliminated most eccentricity variation from the structure-from-motion data, there was some evidence that E(2) increases as accuracy decreases in the shape-from-texture task, suggesting that there may be more than one eccentricity-dependent limitation on performance in this task.

A gradient based heuristic for the perception of 3D shape from texture

A gradient based heuristic for the perception of 3D shape from texture

3D shape perception from combined depth cues in human visual cortex

Our perception of the world's three-dimensional (3D) structure is critical for object recognition, navigation and planning actions. To accomplish this, the brain combines different types of visual information about depth structure, but at present, the neural architecture mediating this combination remains largely unknown. Here, we report neuroimaging correlates of human 3D shape perception from the combination of two depth cues. We measured fMRI responses while observers judged the 3D structure of two sequentially presented images of slanted planes defined by binocular disparity and perspective. We compared the behavioral and fMRI responses evoked by changes in one or both of the depth cues. fMRI responses in extrastriate areas (hMT+/V5 and lateral occipital complex), rather than responses in early retinotopic areas, reflected differences in perceived 3D shape, suggesting 'combined-cue' representations in higher visual areas. These findings provide insight into the neural circuits engaged when the human brain combines different information sources for unified 3D visual perception.