The cospecification of the shape and material properties of light permeable materials

Abstract

The problem of extracting the three-dimensional (3D) shape and material properties of surfaces from images is considered to be inherently ill posed. It is thought that a priori knowledge about either 3D shape is needed to infer material properties, or knowledge about material properties are needed to derive 3D shape. Here, we show that there is information in images that cospecify both the material composition and 3D shape of light permeable (translucent) materials. Specifically, we show that the intensity gradients generated by subsurface scattering, the shape of self-occluding contours, and the distribution of specular reflections covary in systematic ways that are diagnostic of both the surface's 3D shape and its material properties. These sources of image covariation emerge from being causally linked to a common environmental source: 3D surface curvature. We show that these sources of covariation take the form of "photogeometric constraints," which link variations in intensity (photometric constraints) to the sign and direction of 3D surface curvature (geometric constraints). We experimentally demonstrate that this covariation generates emergent cues that the visual system exploits to derive the 3D shape and material properties of translucent surfaces and demonstrate the potency of these cues by constructing counterfeit images that evoke vivid percepts of 3D shape and translucency. The concepts of covariation and cospecification articulated herein suggest a principled conceptual path forward for identifying emergent cues that can be used to solve problems in vision that have historically been assumed to be ill posed.