3D Shape Perception Research Articles

3D shape perception from combined depth cues in human visual cortex

Our perception of the world's three-dimensional (3D) structure is critical for object recognition, navigation and planning actions. To accomplish this, the brain combines different types of visual information about depth structure, but at present, the neural architecture mediating this combination remains largely unknown. Here, we report neuroimaging correlates of human 3D shape perception from the combination of two depth cues. We measured fMRI responses while observers judged the 3D structure of two sequentially presented images of slanted planes defined by binocular disparity and perspective. We compared the behavioral and fMRI responses evoked by changes in one or both of the depth cues. fMRI responses in extrastriate areas (hMT+/V5 and lateral occipital complex), rather than responses in early retinotopic areas, reflected differences in perceived 3D shape, suggesting 'combined-cue' representations in higher visual areas. These findings provide insight into the neural circuits engaged when the human brain combines different information sources for unified 3D visual perception.

A new look at binocular stereopsis

We report a new phenomenon, which illustrates that the role of binocular disparity in 3D shape perception critically depends on whether the parts are interpreted as belonging to a single object. The nature of this phenomenon was studied in four experiments. In the first two experiments the subjects were shown a sequence of stereoscopic images of a cube, in which binocular disparity indicated that the individual parts move towards or away from one eye. However, when the parts of the cube were perceived as elements of a single object, they appeared to move in a rigid fashion and the direction of motion was orthogonal to that predicted by the binocular disparities. The third experiment generalized these results to more complex polyhedra. The last experiment showed that constraints related to motion, such as rigidity, are important, but not critical for this phenomenon to occur. All these results imply that the interpretation as to what corresponds to a single object affects the importance (weight) of binocular disparity and may even eliminate its contribution altogether; the percept of a 3D shape is dominated by a priori constraints, and depth cues play a secondary role.

Three-dimensional shape from non-homogeneous textures: Carved and stretched surfaces

We examined the perception of 3D shape for surfaces folded, carved, or stretched out of textured materials. The textures were composed of sums of sinusoidal gratings or of circular dots, and were designed to differentiate between orientation and frequency information present in perspective images of the surfaces. Correct perception of concavities, convexities, saddles, and slants required the visibility of signature patterns of orientation modulations. These patterns were identical to those identified previously for developable surfaces (A. Li & Q. Zaidi, 2000; Q. Zaidi & L. Li, 2002), despite the fact that textures were statistically homogeneous on developable surfaces but not on carved or stretched surfaces. Frequency modulations in the image were interpreted as cues to distance from the observer, which led to weak but qualitatively correct percepts for some carved and stretched surfaces but to misperceptions for others, similar to the misperceptions for developable surfaces (A. Li & Q. Zaidi, 2003). Irrespective of whether texture on the surface is homogeneous or non-homogeneous, similar neural modules can be used to locate signature orientation modulations and thus extract shape from texture cues.

Specular reflections and the perception of shape

Many materials, including leaves, water, plastic, and chrome exhibit specular reflections. It seems reasonable that the visual system can somehow exploit specular reflections to recover three-dimensional (3D) shape. Previous studies (e.g., J. T. Todd & E. Mingolla, 1983; J. F. Norman, J. T. Todd, & G. A. Orban, 2004) have shown that specular reflections aid shape estimation, but the relevant image information has not yet been isolated. Here we explain how specular reflections can provide reliable and accurate constraints on 3D shape. We argue that the visual system can treat specularities somewhat like textures, by using the systematic patterns of distortion across the image of a specular surface to recover 3D shape. However, there is a crucial difference between textures and specularities: In the case of textures, the image compressions depend on the first derivative of the surface depth (i.e., surface orientation), whereas in the case of specularities, the image compressions depend on the second derivative (i.e., surfaces curvatures). We suggest that this difference provides a cue that can help the visual system distinguish between textures and specularities, even when present simultaneously. More importantly, we show that the dependency of specular distortions on the second derivative of the surface leads to distinctive fields of image orientation as the reflected world is warped across the surface. We find that these "orientation fields" are (i) diagnostic of 3D shape, (ii) remain surprisingly stable when the world reflected in the surface is changed, and (iii) can be extracted from the image by populations of simple oriented filters. Thus the use of specular reflections for 3D shape perception is both easier and more reliable than previous computational work would suggest.

The effects of camera and viewing angles on the perception of 3D shape from texture

The effects of camera and viewing angles on the perception of 3D shape from texture

On the role of phase information in the perception of 3D shape from texture

On the role of phase information in the perception of 3D shape from texture

Using motion to illustrate static 3D shape - kinetic visualization

In this paper, we present a novel visualization technique-kinetic visualization-that uses motion along a surface to aid in the perception of 3D shape and structure of static objects. The method uses particle systems, with rules such that particles flow over the surface of an object to not only bring out, but also attract attention to information on a shape that might not be readily visible with a conventional rendering method which uses lighting and view changes. Replacing still images with animations in this fashion, we demonstrate with both surface and volumetric models in the accompanying videos that, in many cases, the resulting visualizations effectively enhance the perception of three-dimensional shape and structure. We also describe how, for both types of data, a texture-based representation of this motion can be used for interactive visualization using PC graphics hardware. Finally, the results of a user study that we have conducted are presented, which show evidence that the supplemental motion cues can be helpful.

Using Motion to Illustrate Static 3D Shape- Kinetic Visualization

In this paper, we present a novel visualization technique-kinetic visualization-that uses motion along a surface to aid in the perception of 3D shape and structure of static objects. The method uses particle systems, with rules such that particles flow over the surface of an object to not only bring out, but also attract attention to information on a shape that might not be readily visible with a conventional rendering method which uses lighting and view changes. Replacing still images with animations in this fashion, we demonstrate with both surface and volumetric models in the accompanying videos that, in many cases, the resulting visualizations effectively enhance the perception of three-dimensional shape and structure. We also describe how, for both types of data, a texture-based representation of this motion can be used for interactive visualization using PC graphics hardware. Finally, the results of a user study that we have conducted are presented, which show evidence that the supplemental motion cues can be helpful.

Generic and non-generic conditions for the perception of surface shape from texture

Li and Zaidi (Vision Research 40 (2000) 217; 41 (2001) 1519) have recently argued that there are two necessary conditions for the perception of 3D shape from texture: (1) the texture pattern must have a disproportionate amount of energy along directions of principal curvature; and (2) the surface must be viewed with a noticeable amount of perspective. In the present article we present evidence that these conclusions are only valid under a limited set of non-generic viewing conditions. Other relevant factors that need to be considered in this context include the distribution of curvature on an object's surface and the set of possible viewing directions from which it can be observed. For generic viewing directions and patterns of curvature, the perception of surface curvature from texture is only minimally affected by the orientation spectrum of the texture pattern or the amount perspective in its optical projection. Li and Zaidi (Vision Research 41 (2001) 1519) have also identified two characteristic patterns of image contours, which they claim to be the only possible source of information within textured images for determining the direction of surface slant or the sign of surface curvature. In the present article we attempt to show that these characteristic patterns can only arise in natural vision for a limited set of non-generic viewing directions. We also review several other factors that can influence the perceived direction of slant or the perceived sign of curvature, which have been identified previously by other investigators.

Selectivity of the parietal visual neurones in 3D orientation of surface of stereoscopic stimuli.

We identified a group of neurones in the posterior parietal cortex of the monkey that responded preferentially to a flat stimulus in a particular orientation on a 3D computer graphics display. We designated these neurones surface-orientation selective (SOS) neurones since they showed tuning to the 3D orientation of the surface. Almost all of the SOS neurones were sensitive to binocular disparity and their responses increased with the width or length of the stimulus, but the responses of only some of them decreased with increasing thickness of the stimulus. These results suggest that SOS neurones extract surface orientation signals from the binocular disparity signals and play an important role in the perception of 3D shape and the visual guidance of hand movement.

Active vision and the identification of three-dimensional shape

We examined the performance of human observers to identify three-dimensional (3D) shape from motion induced by exploratory head movements. Subjects categorized 3D quadratic surfaces with randomly chosen shape but with a fixed amount of curvature in one of eight shape categories. Human observers had their best performance with convex parabolic and concave parabolic shapes, but had more difficulties in identifying the hyperbolic shapes. The identification of 3D shape was not significantly influenced by the amount of curvature. This means that our description of 3D shape closely matches the intuitive notion of shape, and that its use in 3D shape perception tasks is justified.