Perception of color and material properties in complex scenes

Abstract

How do human observers estimate the location, form, and color of objects? Accurate estimation is challenging because the light arriving at the eyes depends not only on object properties, but also on the spectra and spatial layout of the light sources (Nassau, 1983; Foley et al., 1990). How well the visual system separates illuminant and object properties to achieve a stable representation has traditionally been studied under the rubric of color and lightness constancy. Most previous work used very simple stimuli, typically a few diffusely illuminated surfaces arranged perpendicular to the line of sight. Over the past several years, however, there has been an evident increase of interest in expanding the conceptualization of this area to incorporate effects that emerge only for complex, typically three-dimensional, scenes. The current issue features papers that represent various manifestations of this interest. One line of research investigates how the three-dimensional layout of a scene affects the perception of lightness and color. Although the current work has long-standing antecedents (e.g. Mach, 1886/1959; Hochberg and Beck, 1954; Gilchrist, 1980), methodological advances in i) experimentation with real illuminated objects (e.g. Brainard et. al, 1987; Rutherford and Brainard, 2002; Ripamonti et al., 2004; Robilotto and Zaidi, 2004), ii) the use of sophisticated graphics simulations (e.g. Yang and Maloney, 1999; Fleming, Dror, & Adelson, 2003; Boyaci, Maloney, & Hersh, 2003; Delahunt and Brainard, 2004), iii) the design of hybrid systems that combine real objects with image-based graphics and video projection (Ling and Hurlbert, 2004), and iv) psychophysical procedures (Maloney and Yang, 2003; Obein, Knoblauch, & Vienot, 2004) have opened the door for systematic exploration of a wider range of phenomena. Recent papers include work on how well vision compensates for changes in surface orientation (Boyaci et al., 2003; Ripamonti et al., 2004), how effectively it discounts inter-reflections among nearby surfaces (Bloj, Kersten, & Hurlbert, 1999; Doerschner, Boyaci, & Maloney, 2004; Delahunt and Brainard, 2004), and how the visual system effectively estimates the spectral properties and spatial layout of the illuminant in three-dimensional scenes (Kraft & Brainard, 1999; Yang & Maloney, 1999; Boyaci, Maloney, & Hersh, 2003; Bloj et al., 2004; Boyaci, Doerschner, & Maloney, 2004; Khang and Zaidi, 2004). The second thread that leads to papers in the current issue is a focus on the functional utility of color and lightness perception -- the idea that these percepts inform us about the properties of objects rather than those of light spectra. This focus resulted in a renaissance of research in color constancy over the past two decades, with particular progress being made in the development of computational models that explore how, in principle, object surface properties can be estimated from image data. As with the experimental lines, early work focused on simple scene geometries (for reviews see Hurlbert, 1998; Maloney, 1999) but consideration has recently expanded to three-dimensional configurations (Adelson and Pentland, 1996; Yang and Maloney, 1999; Bell and Freeman, 2001; Dror, Willsky, & Adelson, 2004) Of particular interest has been the elaboration of purely computational formulations into parametric models of human performance (e.g. Brainard Brunt, & Speigle, 1997; Brainard Kraft, & Longere, 2003; Boyaci et al., 2003; Doerschner et al., 2004; Boyaci et al., 2004; Bloj et al., 2004), tests of how well the visual system exploits image information identified in computational studies (Yang and Maloney, 2001; Delahunt and Brainard, 2004; Smithson and Zaidi, 2004), investigations of how well the visual system recovers perceptual correlates of material properties other than diffuse surface reflectance, such as gloss and translucency (Lu, Koenderink, & Kappers, 2000; Fleming et al., 2003; Pont & Koenderink, 2003; Obein et al., 2004), as well as how geometric aspects of surface reflectance interact with the perception of shape (Fleming et al., 2003).