Perception of three-dimensional shape from texture is based on patterns of oriented energy

Abstract

This paper presents empirical support for a new observer model of inferring three-dimensional shape from monocular texture cues. By measuring observers’ abilities to estimate the relative three-dimensional curvature along a textured surface from two-dimensional projected images, and concurrently examining the local spectral changes occurring in the projected image for various texture patterns, we have found that correlated changes in oriented energy along lines corresponding to the lines of maximum and minimum curvature of the surface are crucial for conveying the three-dimensional shape of the surface. Energy along these lines of maximum and minimum curvature can be used to compute the orientation of local surface patches. Texture patterns consisting of simple and complex sinusoidal gratings and plaids, and filtered noise were drawn onto a surface that was corrugated sinusoidally in depth about the horizontal axis and projected in perspective onto an image plane. The perceived relative surface curvature was reconstructed from measurements of local ordinal depth around a central fixation point at 12 different phases of the corrugation. Our results show that: (1) it is neither necessary nor sufficient to identify individual texture elements or texture gradients in order to extract the shape of the surface; (2) one-dimensional frequency modulation is insufficient for conveying complex three-dimensional shape. (3) Veridical ordinal depth is seen only when the projected pattern contains changes in oriented energy along lines corresponding to projected lines of maximum curvature of the surface. (4) For a surface corrugated in depth about the horizontal axis, this pattern of oriented energy arises from energy along the vertical direction in the global Fourier transform of the pre-corrugated pattern. (5) Local orientation changes across lines of minimum curvature can be also critical for conveying shape. (6) These correlated orientation changes along lines of maximum and minimum curvature are entirely lost in parallel projection. Hence texture is a useful cue for shape if the image is a perspective projection. (7) Only some natural textures will provide sufficient monocular cues to support veridical shape inferences, and this can be predicted from their global Fourier transforms.