Shape from contours constrains shape from shading

Abstract

When light illuminates a 3D body it generates a luminance distribution (shading pattern) that can be calculated using the irradiance equation. However, reconstructing the 3D shape from a given luminance distribution is not trivial. Mallot (2000) showed that for 1D luminance distributions the irradiance equation can be inverted and solved for the depth profile, in some cases analytically. The solutions are not unique but involve families of curves. Therefore the question arises how the visual system ‘chooses’ particular shapes as perceived depth profiles. One possible constraint is the shape of the contour of the shading pattern. On the 2009 VSS I reported that this shape can indeed affect the perceived depth profile. Here I report new mathematical analyses and a new experiment using as stimuli three-cycle sinusoidal luminance distributions bounded by 12 differently shaped contours. The tasks of the 102 subjects for each of the 12 stimuli were a) to choose the corresponding perceived depth profile from a presented set of 17 profiles depicted as outline sketches without shadings, b) to rate its depth extent on a scale from 0 to 3, and c) to judge its illumination direction by choosing from 5 possibilities spanning the range from 0 to 180 degrees. The results showed surprisingly strong, generally consistent and qualitatively diverse effects of different shapes of contours of otherwise identical shading patterns on their perceived 3D shapes. Many aspects of the perceptual effects can be accounted for by a mathematical result by Koenderink (1984), who showed that features of the local shape of smooth 3D bodies can be deduced from the 2D shape of their bounding contours. When contoured shaded regions generate depth percepts, the effects of contour shapes (contour constraints) interact with the effects provided by the interior structure of the shading patterns (shading constraints).