Perception Of Three-dimensional Shape Research Articles

Perception of 3D shape integrates intuitive physics and analysis-by-synthesis.

Many surface cues support three-dimensional shape perception, but humans can sometimes still see shape when these features are missing-such as when an object is covered with a draped cloth. Here we propose a framework for three-dimensional shape perception that explains perception in both typical and atypical cases as analysis-by-synthesis, or inference in a generative model of image formation. The model integrates intuitive physics to explain how shape can be inferred from the deformations it causes to other objects, as in cloth draping. Behavioural and computational studies comparing this account with several alternatives show that it best matches human observers (total n = 174) in both accuracy and response times, and is the only model that correlates significantly with human performance on difficult discriminations. We suggest that bottom-up deep neural network models are not fully adequate accounts of human shape perception, and point to how machine vision systems might achieve more human-like robustness.

Visual perception: Colour brings shape into stark relief

Recent research has uncovered a surprising new role of colour in the perception of three-dimensional shape. The brain is exquisitely sensitive to visual patterns emerging from the way different wavelengths interact with surfaces.

The role of color in the perception of three-dimensional shape

SummaryThe human visual system can derive information about three-dimensional (3D) shape from the structure of light reflected by surfaces. Most research on single static images has focused on the 3D shape information contained in variations of brightness caused by interactions between the illumination and local surface orientation (“shading”).1, 2, 3, 4, 5, 6 Although color can enhance the recovery of surface shading when color and brightness vary independently,7, 8, 9 there is no evidence that color alone provides any information about 3D shape. Here, we show that the wavelength-dependent reflectance of chromatic materials provides information about the 3D shape of translucent materials. We show that different wavelengths of light undergo varying degrees of subsurface light transport, which generates multiple forms of spatial structure: wavelengths that are weakly reflected generate shading-like image structure, linked to 3D surface orientation, whereas wavelengths that penetrate more deeply into the material are primarily constrained by the direction of surface curvature (convexities and concavities).10 Psychophysical experiments demonstrate that the enhanced perception of 3D shape in chromatic translucent surfaces arises from the shading structure generated by weakly reflected wavelengths, which, in turn, generates correlated spatial variations in saturation. These results demonstrate a new functional role for color in the perception of the 3D shape of translucent materials.

On the Ambient Optic Array: James Gibson's Insights About the Phenomenon of Chiaroscuro.

In 1966, James Gibson first presented his theory of the ambient optic array, and he proposed a new field of ecological optics that he hoped would advance our knowledge on this topic. This study will consider how his ideas have largely come to fruition over the past 50 years. It reviews the research on the visual perception of three-dimensional shape from shading, the effects of ambient light from surface interreflections on observers’ perceptions, the perception of the light field, and the perception of surface materials. Finally, it also considers Gibson’s impact on these developments.

Binocular stereo acuity affects monocular three-dimensional shape perception in patients with strabismus

Background/aimsTo evaluate the perception of three-dimensional (3D) shape in patients with strabismus and the contributions of stereopsis and monocular cues to this perception.MethodsTwenty-one patients with strabismus with and 20 without...

An empirical assessment of the impact of the light direction on the relief inversion effect in shaded relief maps: NNW is better than NW

ABSTRACTRelief inversion (or terrain reversal) effect is a well-known phenomenon in cartography that occurs when shadow is the main depth cue for three-dimensional shape perception. Light direction has been suggested as the main cause of this effect. However, the prevalence of relief inversion effect with regard to the changing light direction is currently not established, and there is little empirical evidence on this subject. This article systematically assesses the influence of light direction on the accuracy of landform perception in shaded relief maps (SRM). In a controlled experiment, 27 participants were asked to identify concave and convex landforms in 128 SRMs using a 5-point Likert scale where answers varied from clearly a valley to clearly a ridge. Eight different scenes were illuminated from 16 light directions to obtain the 128 SRMs. Our findings clearly demonstrate that incident light at 337.5° north-northwest (NNW) yields the highest accuracy and confidence ratings in landform identification among the investigated light directions; and leads to higher accuracy scores than at the 315° (NW) which is conventionally used in SRMs. Thus, we propose an update to this convention and recommend the light source to be placed at 337.5° when creating SRMs.

Specular Image Structure Modulates the Perception of Three-Dimensional Shape

Retinal images are produced by interactions between a surface's 3D shape, material properties, and surrounding light field. In order to recover the 3D geometry of a surface, the visual system must somehow separate aspects of image structure generated by a surface's shape from structure generated by its material properties or the light field in which it is embedded. Attributing image structure to the wrong physical source would cause the visual system to interpret changes in one physical property (such as reflectance) as changes in another (such as shape). Many previous studies have shown that the visual system does not conflate image structure generated by specular reflectance with 3D shape, but they did not assess the physical conditions where it would be computationally most difficult to disentangle these different sources of image structure. Here, we show that varying the specular roughness and curvature of surfaces embedded in natural light fields can strongly modulate perceived shape. Despite the complexity of these interactions, we show how an image's gradient structure mediates its interpretation as a specular reflection or a change in 3D shape. Our findings provide a coherent explanation of when and why specular reflections impact perceived shape and reveal how the static surface properties, simplified light fields, and experimental methods used in previous studies may explain their inconsistent results.

Binocular Eye Movements Evoked by Self-Induced Motion Parallax

Perception often triggers actions, but actions may sometimes be necessary to evoke percepts. This is most evident in the recovery of depth by self-induced motion parallax. Here we show that depth information derived from one's movement through a stationary environment evokes binocular eye movements consistent with the perception of three-dimensional shape. Human subjects stood in front of a display and viewed a simulated random-dot sphere presented monocularly or binocularly. Eye movements were recorded by a head-mounted eye tracker, while head movements were monitored by a motion capture system. The display was continuously updated to simulate the perspective projection of a stationary, transparent random dot sphere viewed from the subject's vantage point. Observers were asked to keep their gaze on a red target dot on the surface of the sphere as they moved relative to the display. The movement of the target dot simulated jumps in depth between the front and back surfaces of the sphere along the line of sight. We found the subjects' eyes converged and diverged concomitantly with changes in the perceived depth of the target. Surprisingly, even under binocular viewing conditions, when binocular disparity signals conflict with depth information from motion parallax, transient vergence responses were observed. These results provide the first demonstration that self-induced motion parallax is sufficient to drive vergence eye movements under both monocular and binocular viewing conditions.

Contribution of Inferior Temporal and Posterior Parietal Activity to Three-Dimensional Shape Perception

One of the fundamental goals of neuroscience is to understand how perception arises from the activity of neurons in the brain. Stereopsis is a type of three-dimensional (3D) perception that relies on two slightly different projections of the world onto the retinas of the two eyes, i.e., binocular disparity. Neurons selective for curved surfaces defined by binocular disparity may contribute to the perception of an object's 3D structure. Such neurons have been observed in both the anterior lower bank of the superior temporal sulcus (TEs, part of the inferior temporal cortex [IT]) and the anterior intraparietal area (AIP). However, the specific contributions of IT and AIP to depth perception remain unknown. We simultaneously recorded multiunit activity in IT and AIP while monkeys discriminated between concave and convex 3D shapes. We observed a correlation between the neural activity and behavioral choice that arose early and during perceptual decision formation in IT but later and after perceptual decision formation in AIP. These results suggest a role for IT, but not AIP, in 3D shape discrimination. Furthermore, the results demonstrate that similar neuronal stimulus selectivities in two areas do not imply a similar function.

Decoding depth order and three-dimensional shape perception from human cortical activity of dorsal and ventral areas

Decoding depth order and three-dimensional shape perception from human cortical activity of dorsal and ventral areas

Haptic perception disambiguates visual perception of 3D shape

We studied the influence of haptics on visual perception of three-dimensional shape. Observers were shown pictures of an oblate spheroid in two different orientations. A gauge-figure task was used to measure their perception of the global shape. In the first two sessions only vision was used. The results showed that observers made large errors and interpreted the oblate spheroid as a sphere. They also misinterpreted the rotated oblate spheroid for a prolate spheroid. In two subsequent sessions observers were allowed to touch the stimulus while performing the task. The visual input remained unchanged: the observers were looking at the picture and could not see their hands. The results revealed that observers perceived a shape that was different from the vision-only sessions and closer to the veridical shape. Whereas, in general, vision is subject to ambiguities that arise from interpreting the retinal projection, our study shows that haptic input helps to disambiguate and reinterpret the visual input more veridically.

Neural basis of 3-D shape aftereffects

We used selective adaptation to identify the neural mechanisms responsible for 3-D shape perception from orientation flows in retinal images [Li, A., & Zaidi, Q. (2000). Perception of three-dimensional shape from texture is based on patterns of oriented energy. Vision Research 40 (2), 217–242)]. Three-dimensional shape adaptation could involve stages from photoreceptors to non-oriented retinal cells, oriented cells in striate cortex, and extra-striate cells that respond to 3-D slants. To psychophysically isolate the relevant stage, we used 3-D adapting stimuli created from real and illusory orientations, and test stimuli different from the adapting stimuli in phases and frequencies. The results showed that mechanisms that adapt to 3-D shapes combine real and illusory 2-D orientation information over a range of spatial frequencies.

The effects of viewing angle, camera angle, and sign of surface curvature on the perception of three-dimensional shape from texture

Computational models for determining three-dimensional shape from texture based on local foreshortening or gradients of scaling are able to achieve accurate estimates of surface relief from an image when it is observed from the same visual angle with which it was photographed or rendered. These models produce conflicting predictions, however, when an image is viewed from a different visual angle. An experiment was performed to test these predictions, in which observers judged the apparent depth profiles of hyperbolic cylinders under a wide variety of conditions. The results reveal that the apparent patterns of relief from texture are systematically underestimated; convex surfaces appear to have greater depth than concave surfaces, large camera angles produce greater amounts of perceived depth than small camera angles, and the apparent depth-to-width ratio for a given image of a surface is greater for small viewing angles than for large viewing angles. Because these results are incompatible with all existing computational models, a new model is presented based on scaling contrast that can successfully account for all aspects of the data.

Three-dimensional shape perception from chromatic orientation flows

The role of chromatic information in 3-D shape perception is controversial. We resolve this controversy by showing that chromatic orientation flows are sufficient for accurate perception of 3-D shape. Chromatic flows required less cone contrast to convey shape than did achromatic flows, thus ruling out luminance artifacts as a problem. Luminance artifacts were also ruled out by a protanope's inability to see 3-D shape from chromatic flows. Since chromatic orientation flows can only be extracted from retinal images by neurons that are responsive to color modulations and selective for orientation, the psychophysical results also resolve the controversy over the existence of such neurons. In addition, we show that identification of 3-D shapes from chromatic flows can be masked by luminance modulations, indicating that it is subserved by orientation-tuned neurons sensitive to both chromatic and luminance modulations.

３次元形状手がかりに基づく陰影からの表面反射特性知覚

Shading in two-dimensional images produces a compelling perception of three-dimensional shape. When mechanisms of perception are considered from an engineering standpoint, shape-from-shading is known to be an ill-posed problem that is difficult to solve. Constraints are needed to solve this kind of problem. Although estimation of reflectance properties from shading is also considered an ill-posed problem, mechanisms of estimation by the visual system are still poorly understood. Therefore, we proposed a psychological experiment intended to determine whether the visual system can estimate surface reflectance properties from shading. In the experiment, we expect to use contours as a cue for perceiving three-dimensional structures. From a preliminary experiment we obtained data that did not rule out that the ability of the visual system to estimate surface reflectance from shading.

Perception of three-dimensional shape from specular highlights, deformations of shading, and other types of visual information.

There have been numerous computational models developed in an effort to explain how the human visual system analyzes three-dimensional (3D) surface shape from patterns of image shading, but they all share some important limitations. Models that are applicable to individual static images cannot correctly interpret regions that contain specular highlights, and those that are applicable to moving images have difficulties when a surface moves relative to its sources of illumination. Here we describe a psychophysical experiment that measured the sensitivity of human observers to small differences of 3D shape over a wide variety of conditions. The results provide clear evidence that the presence of specular highlights or the motions of a surface relative to its light source do not pose an impediment to perception, but rather, provide powerful sources of information for the perceptual analysis of 3D shape.

The perception of doubly curved surfaces from anisotropic textures.

Most existing computational models of the visual perception of three-dimensional shape from texture are based on assumed constraints about how texture is distributed on visible surfaces. The research described in the present article was designed to investigate how violations of these assumptions influence human perception. Observers were presented with images of smoothly curved surfaces depicted with different types of texture, whose distribution of surface markings could be both anisotropic and inhomogeneous. Observers judged the pattern of ordinal depth on each object by marking local maxima and minima along designated scan lines. They also judged the apparent magnitudes of relative depth between designated probe points on the surface. The results revealed a high degree of accuracy and reliability in all conditions, except for a systematic underestimation of the overall magnitude of surface relief. These findings suggest that human perception of three-dimensional shape from texture is much more robust than would be reasonable to expect based on current computational models of this phenomenon.

Using Motion to Illustrate Static 3D Shape- Kinetic Visualization

In this paper, we present a novel visualization technique-kinetic visualization-that uses motion along a surface to aid in the perception of 3D shape and structure of static objects. The method uses particle systems, with rules such that particles flow over the surface of an object to not only bring out, but also attract attention to information on a shape that might not be readily visible with a conventional rendering method which uses lighting and view changes. Replacing still images with animations in this fashion, we demonstrate with both surface and volumetric models in the accompanying videos that, in many cases, the resulting visualizations effectively enhance the perception of three-dimensional shape and structure. We also describe how, for both types of data, a texture-based representation of this motion can be used for interactive visualization using PC graphics hardware. Finally, the results of a user study that we have conducted are presented, which show evidence that the supplemental motion cues can be helpful.

Using motion to illustrate static 3D shape - kinetic visualization

In this paper, we present a novel visualization technique-kinetic visualization-that uses motion along a surface to aid in the perception of 3D shape and structure of static objects. The method uses particle systems, with rules such that particles flow over the surface of an object to not only bring out, but also attract attention to information on a shape that might not be readily visible with a conventional rendering method which uses lighting and view changes. Replacing still images with animations in this fashion, we demonstrate with both surface and volumetric models in the accompanying videos that, in many cases, the resulting visualizations effectively enhance the perception of three-dimensional shape and structure. We also describe how, for both types of data, a texture-based representation of this motion can be used for interactive visualization using PC graphics hardware. Finally, the results of a user study that we have conducted are presented, which show evidence that the supplemental motion cues can be helpful.

Neural correlates of structure-from-motion perception in macaque V1 and MT.

Structure-from-motion (SFM) is the perception of three-dimensional shape from motion cues. We used a bistable SFM stimulus, which can be perceived in one of two different ways, to study how neural activity in cortical areas V1 and MT is related to SFM perception. Monkeys performed a depth-order task, where they indicated in which direction the front surface of a rotating SFM cylinder display was moving. To prevent contamination of the neural data because of eye position effects, all experiments with significant effects of radius, vergence, and velocity were excluded. As expected, the activity of approximately 50% of neurons in V1 and approximately 80% of neurons in MT is affected by the stimulus. Furthermore, the activity of 20% of neurons in area V1 is modulated with the percept. This proportion is higher in MT, where the activity of >60% of neurons is modulated with the percept. In both areas, this perceptual modulation occurs only in neurons with activity that is also affected by the stimulus. The perceptual modulation is not correlated with neural tuning properties in area V1, but it is in area MT. Together, these results suggest that V1 is not directly involved in the generation of the SFM percept, whereas MT is. The perceptual modulation in V1 may be attributable to top-down feedback from MT.