3D Shape Perception Research Articles

A supramodal thorough account of the Molyneux question

In this paper, we want to tackle the Molyneux question thoroughly, by addressing it in terms of both ordinary perception and pictorial perception: if a congenitally blind person recovered sight, could she recognize visually the 3D shapes she already recognized tactilely, both when such shapes are given to her directly and when they are given to her pictorially, i.e., as depicted shapes? We want to claim that empirical evidence suggests that the question can be positively answered in both cases. For in the first case, such evidence shows that perception of 3D shapes is supramodal; namely, it can be equivalently achieved in different sense modalities, notably touch and vision, independently of the sensory input such shapes are accessed. While in the second case, such evidence shows that one can satisfy both in vision and in touch the condition for depicted shapes, which are typically not where the perceiver is, to be grasped by that perceiver in a picture’s subject, i.e., what the picture presents. This condition states that the picture’s vehicle, i.e., the typically 2D physical basis of a picture, is enriched by adding to its properties the 3D grouping properties that allow for a figure/ground segmentation to be performed in that vehicle’s elements.

A tutorial on the physics of light and image shading.

This paper provides an overview of the many different ways that light interacts with surfaces in the natural environment to provide useful information for visual perception. It begins with a discussion of how the concept of light has evolved over the course of human history. It then considers a wide variety of optical phenomena including Lambert's laws of illumination, the effects of microscopic surface structure on patterns of reflection, the bidirectional reflectance distribution function, the refraction of transmitted light, chromatic dispersion, thin film interference, sub-surface scattering, the Fresnel effects, indirect illumination from multiple reflections, caustics, and the structure of the light field. The primary goal of this discussion is to provide the necessary background information to help students and young researchers more easily understand the scientific literature on the perception of 3D shape and material properties from patterns of image shading.

Perceptual Biases in the Interpretation of Non-Rigid Shape Transformations from Motion.

Most existing research on the perception of 3D shape from motion has focused on rigidly moving objects. However, many natural objects deform non-rigidly, leading to image motion with no rigid interpretation. We investigated potential biases underlying the perception of non-rigid shape interpretations from motion. We presented observers with stimuli that were consistent with two qualitatively different interpretations. Observers were shown a two-part 3D object with the smaller part changing in length dynamically as the whole object rotated back and forth. In two experiments, we studied the misperception (i.e., perceptual reinterpretation) of the non-rigid length change to a part. In Experiment 1, observers misperceived this length change as a part orientation change (i.e., the smaller part was seen as articulating with respect to the larger part). In Experiment 2, the stimuli were similar, except the silhouette of the part was visible in the image. Here, the non-rigid length change was reinterpreted as a rigidly attached part with an "illusory" non-orthogonal horizontal angle relative to the larger part. We developed a model that incorporated this perceptual reinterpretation and could predict observer data. We propose that the visual system may be biased towards part-wise rigid interpretations of non-rigid motion, likely due to the ecological significance of movements of humans and other animals, which are generally constrained to move approximately part-wise rigidly. That is, not all non-rigid deformations are created equal: the visual systems' prior expectations may bias the system to interpret motion in terms of biologically plausible shape transformations.

Active shape reconstruction using a novel visuotactile palm sensor

Tactile sensing enables high-precision 3D shape perception when vision is limited. However, tactile-based shape reconstruction remains a challenging problem. In this paper, a novel visuotactile sensor, GelStereo Palm 2.0, is proposed to better capture 3D contact geometry. Leveraging the dense tactile point cloud captured by GelStereo Palm 2.0, an active shape reconstruction pipeline is presented to achieve accurate and efficient 3D shape reconstruction on irregular surfaces. GelStereo Palm 2.0 achieves a spatial resolution of 1.5 mm and a reconstruction accuracy of 0.3 mm. The accuracy of the proposed active shape reconstruction pipeline reaches 2.3 mm within 18 explorations. The proposed method has potential applications in the shape reconstruction of transparent or underwater objects.

Toward a theory of perspective perception in pictures.

This paper reviews projection models and their perception in realistic pictures, and proposes hypotheses for three-dimensional (3D) shape and space perception in pictures. In these hypotheses, eye fixations, and foveal vision play a central role. Many past theories and experimental studies focus solely on linear perspective. Yet, these theories fail to explain many important perceptual phenomena, including the effectiveness of nonlinear projections. Indeed, few classical paintings strictly obey linear perspective, nor do the best distortion-avoidance techniques for wide-angle computational photography. The hypotheses here employ a two-stage model for 3D human vision. When viewing a picture, the first stage perceives 3D shape for the current gaze. Each fixation has its own perspective projection, but, owing to the nature of foveal and peripheral vision, shape information is obtained primarily for a small region of the picture around the fixation. As a viewer moves their eyes, the second stage continually integrates some of the per-gaze information into an overall interpretation of a picture. The interpretation need not be geometrically stable or consistent over time. It is argued that this framework could explain many disparate pictorial phenomena, including different projection styles throughout art history and computational photography, while being consistent with the constraints of human 3D vision. The paper reviews open questions and suggests new studies to explore these hypotheses.

Extraction of three-dimensional shapes in glaucoma patients in response to monocular depth cues.

To assess the impact of glaucoma on perceiving three-dimensional (3D) shapes based on monocular depth cues. Clinical observational study. Twenty glaucoma patients, subjected to binocular visual-field sensitivity (binocular-VFS) tests using a Humphrey Visual Field Analyzer, and 20 age-matched healthy volunteers, underwent two tasks: identifying the nearest vertex of a 3D shape using monocular shading (3D-SfS), texture (3D-SfT), or motion (3D-SfM) cues, and distinguishing elementary one-dimensional (1D) features of these cues. The association of the visual-field index (VFI) of binocular-VFS with 3D shape perception in glaucoma patients was also examined. Glaucoma patients demonstrated reduced accuracy in distinguishing 1D luminance brightness and a larger "error-in-depth" between the perceived and actual depths for 3D-SfM and 3D-SfS compared to healthy volunteers. Six glaucoma patients with a 100% VFI for binocular-VFS exhibited a similar error-in-depth to the other fourteen glaucoma patients; they had a larger error-in-depth for 3D-SfM compared to healthy volunteers. No correlation between the error-in-depth values and the VFI values of binocular-VFS was observed. The 3D shape perception in glaucoma patients varies based on the depth cue's characteristics. Impaired 1D discrimination and larger thresholds for 3D-SfM in glaucoma patients with a 100% VFI for binocular-VFS indicate more pronounced perceptual deficits of lower-level elementary features for 3D-SfS and higher-level visual processing of 3D shapes for 3D-SfM. The effects of the location and degree of binocular visual-field defects on 3D shape perception remain to be elucidated. Our research provides insights into the 3D shape extraction mechanism in glaucoma.

Modality-Independent Effect of Gravity in Shaping the Internal Representation of 3D Space for Visual and Haptic Object Perception.

Visual and haptic perceptions of 3D shape are plagued by distortions, which are influenced by nonvisual factors, such as gravitational vestibular signals. Whether gravity acts directly on the visual or haptic systems or at a higher, modality-independent level of information processing remains unknown. To test these hypotheses, we examined visual and haptic 3D shape perception by asking male and female human subjects to perform a "squaring" task in upright and supine postures and in microgravity. Subjects adjusted one edge of a 3D object to match the length of another in each of the three canonical reference planes, and we recorded the matching errors to obtain a characterization of the perceived 3D shape. The results show opposing, body-centered patterns of errors for visual and haptic modalities, whose amplitudes are negatively correlated, suggesting that they arise in distinct, modality-specific representations that are nevertheless linked at some level. On the other hand, weightlessness significantly modulated both visual and haptic perceptual distortions in the same way, indicating a common, modality-independent origin for gravity's effects. Overall, our findings show a link between modality-specific visual and haptic perceptual distortions and demonstrate a role of gravity-related signals on a modality-independent internal representation of the body and peripersonal 3D space used to interpret incoming sensory inputs.

Perception of 3D shape integrates intuitive physics and analysis-by-synthesis.

Many surface cues support three-dimensional shape perception, but humans can sometimes still see shape when these features are missing-such as when an object is covered with a draped cloth. Here we propose a framework for three-dimensional shape perception that explains perception in both typical and atypical cases as analysis-by-synthesis, or inference in a generative model of image formation. The model integrates intuitive physics to explain how shape can be inferred from the deformations it causes to other objects, as in cloth draping. Behavioural and computational studies comparing this account with several alternatives show that it best matches human observers (total n = 174) in both accuracy and response times, and is the only model that correlates significantly with human performance on difficult discriminations. We suggest that bottom-up deep neural network models are not fully adequate accounts of human shape perception, and point to how machine vision systems might achieve more human-like robustness.

3D shape recovery algorithm from image orientations of textured surfaces.

Previous psychophysical studies have demonstrated that the image orientation of textured surfaces guides human 3D shape perception. However, the accuracy of computational 3D shape reconstruction solely from image orientation requires further study. This paper proposes a 3D shape recovery algorithm from the image orientation of a single textured surface image. The evaluation of the proposed algorithm uses computer-generated textured complex 3D surfaces. The depth correlations between the recovered and true surface shapes achieved or exceeded 0.8, which is similar to the accuracy of human shape perception, as shown in a previous psychophysical study, indicating that the image orientations contain adequate information for 3D shape recovery from textured surface images.

Qualitative perception of 3D shape from patterns of luminance curvature.

A new source of information is proposed for the perception of three-dimensional (3D) shape from shading that identifies surface concavities from the curvature of the luminance field. Two experiments measured the abilities of human observers to identify concavities on smoothly curved shaded surfaces depicted with several different patterns of illumination and several different material properties. Observers were required to identify any apparent concavities along designated cross sections of the depicted objects and to mark each concavity with an adjustable dot. To analyze the results, we computed both the surface curvature and the luminance curvature along each image cross section. The results revealed that most responses were in concave regions of the luminance profiles, although they were often shifted in phase relative to the curvature of the depicted surfaces. This pattern of performance was surprisingly robust over large changes in the pattern of illumination or surface material properties. Our analysis predicts that observers should make false alarm responses in regions where a luminance concavity does not correspond to a surface concavity, and our empirical results confirm that prediction.

Are mirror-symmetric objects of special importance for 3D shape perception? A reply to Sawada and Pizlo (2022).

Yu, Todd, and Petrov (2021) and Yu, Petrov, and Todd (2021) investigated failures of shape constancy that occur when objects are viewed stereoscopically at different distances. Although this result has been reported previously with simple objects such as pyramids or cylinders, we examined more complex objects with bilateral symmetry to test the claim by Li, Sawada, Shi, Kwon, and Pizlo (2011) that the perception of those objects is veridical. Sawada and Pizlo (2022) offer several criticisms of our experiments, but they seem to suggest that the concept of shape is defined by what is computable by their model. If stimuli are used that cannot be discriminated by their model, they are dismissed as degenerate, and tasks that cannot be performed by their model are assumed to be based on something other than shape. This allows them to disregard empirical evidence that is inconsistent with their model. We argue, in contrast, that all reliable aspects of shape perception are deserving of explanation. We also argue that there are many different attributes of shape and many different sources of information about shape that may be relevant in different contexts. It is unlikely that all of them can be explained by a single model.

Testing a formal theory of perception is not easy: Comments on Yu, Todd & Petrov (2021) and Yu, Petrov & Todd (2021).

Yu, Todd, and Petrov (2021, Journal of Vision) and their follow-up study (Yu, Petrov, & Todd, 2021, i-Perception) aimed at evaluating the role of three-dimensional (3D) symmetry in binocular shape perception by comparing their experimental data to predictions they derived from our computational models. We point out in this note that their predictions were incorrect, so their studies can neither reject nor support our models of 3D shape perception. We explain (1) the role of the data and the constraints in solving ill-posed inverse problems, (2) the role of binocular depth-order, as opposed to binocular depth-intervals in shape perception, (3) the nature and the effect of 3D compactness as an a priori constraint, and (4) the implications of the separation of binocular disparity and stereoacuity in the two functional streams in the visual cortex.

The Role of Specular Reflections and Illumination in the Perception of Thickness in Solid Transparent Objects.

Specular reflections and refractive distortions are complex image properties of solid transparent objects, but despite this complexity, we readily perceive the 3D shapes of these objects (e.g., glass and clear plastic). We have found in past work that relevant sources of scene complexity have differential effects on 3D shape perception, with specular reflections increasing perceived thickness, and refractive distortions decreasing perceived thickness. In an object with both elements, such as glass, the two optical properties may complement each other to support reliable perception of 3D shape. We investigated the relative dominance of specular reflection and refractive distortions in the perception of shape. Surprisingly, the ratio of specular reflection to refractive component was almost equal to that of ordinary glass and ice, which promote correct percepts of 3D shape. The results were also explained by the variance in local RMS contrast in stimulus images but may depend on overall luminance and contrast of the surrounding light field.

The role of color in the perception of three-dimensional shape

SummaryThe human visual system can derive information about three-dimensional (3D) shape from the structure of light reflected by surfaces. Most research on single static images has focused on the 3D shape information contained in variations of brightness caused by interactions between the illumination and local surface orientation (“shading”).1, 2, 3, 4, 5, 6 Although color can enhance the recovery of surface shading when color and brightness vary independently,7, 8, 9 there is no evidence that color alone provides any information about 3D shape. Here, we show that the wavelength-dependent reflectance of chromatic materials provides information about the 3D shape of translucent materials. We show that different wavelengths of light undergo varying degrees of subsurface light transport, which generates multiple forms of spatial structure: wavelengths that are weakly reflected generate shading-like image structure, linked to 3D surface orientation, whereas wavelengths that penetrate more deeply into the material are primarily constrained by the direction of surface curvature (convexities and concavities).10 Psychophysical experiments demonstrate that the enhanced perception of 3D shape in chromatic translucent surfaces arises from the shading structure generated by weakly reflected wavelengths, which, in turn, generates correlated spatial variations in saturation. These results demonstrate a new functional role for color in the perception of the 3D shape of translucent materials.

Regions of High Curvature Help to Stabilize the Perception of 3D Shape

Regions of High Curvature Help to Stabilize the Perception of 3D Shape

Just Noticeable Differences in 3D Shape Perception: A Measure of Estimation Noise or Cue Gain?

The present study sought to test two fundamentally distinct interpretations of the Just Noticeable Difference (JND) for depth discrimination. According to the widely accepted Bayesian Maximum Likelihood Estimation (MLE) model of 3D processing, the JND measures the standard deviation of the noise corrupting a depth estimate. The Intrinsic Constraint (IC) model of cue integration alternatively suggests that the JND is the result of task related decision noise rather than noise directly corrupting the depth estimate. According to this account, the JND is inversely proportional to the slope, or gain, of the perceptual function relating physical depth to perceived depth. To test this novel interpretation, we tested depth discrimination with a classic 2-Interval Forced Choice task in which the gains of a fixed standard stimulus and varying comparison stimulus could either have a High or Low value, or, according to Bayesian models, High or Low reliabilities. Bayesian models predict that the JND should depend on both the reliability of the standard and comparison stimuli. However, according to the IC model, the JND only depends on the gain of the comparison stimulus, and therefore there should be no difference between conditions where the comparison has the same gain. Empirical results with texture stimuli closely align with the IC predictions. In conclusion, since JNDs do not measure the magnitude of the noise of 3D estimates as assumed by the Bayesian approach, these findings lead to a radically different interpretation of previous depth discrimination data.

Probing human 3D shape perception with novel, but natural stimuli

Probing human 3D shape perception with novel, but natural stimuli

Red shape, blue shape: political ideology influences the social perception of body shape

Political elections have a profound impact on individuals and societies. Optimal voting is thought to be based on informed and deliberate decisions yet, it has been demonstrated that the outcomes of political elections are biased by the perception of candidates’ facial features and the stereotypical traits voters attribute to these. Interestingly, political identification changes the attribution of stereotypical traits from facial features. This study explores whether the perception of body shape elicits similar effects on political trait attribution and whether these associations can be visualized. In Experiment 1, ratings of 3D body shapes were used to model the relationship between perception of 3D body shape and the attribution of political traits such as ‘Republican’, ‘Democrat’, or ‘Leader’. This allowed analyzing and visualizing the mental representations of stereotypical 3D body shapes associated with each political trait. Experiment 2 was designed to test whether political identification of the raters affected the attribution of political traits to different types of body shapes. The results show that humans attribute political traits to the same body shapes differently depending on their own political preference. These findings show that our judgments of others are influenced by their body shape and our own political views. Such judgments have potential political and societal implications.

Research on mathematical model of characteristic curve of surface perception by PVDF array based on Ferguson function

The traditional 3D shape perception and reconstruction requires the use of relatively expensive 3D scanning equipment to obtain the coordinates of discrete feature points of the 3D shape. The reconstruction accuracy is determined by the scanning accuracy of the 3D scanning device, generally, the accuracy is low. In this study, based on the PVDF piezoelectric film sensing array, the 3D topographic feature point coordinate value is perceived and acquired. According to the feature point coordinate value, the feature curve fitting of the 3D shape is obtained, and the slope information of the 3D topography characteristic curve is obtained. The mathematical model of the feature surface of PVDF array surface topography based on Ferguson curve is studied to realise the surface reconstruction of 3D complex topography. Theoretical research shows that three-dimensional complex topographic reconstruction technique based on Ferguson curve for PVDF piezoelectric film array surface topography perception has the advantages of simple principle, easy implementation and high precision of shape reconstruction. It can develop intelligent structure with three-dimensional complex shape perception function.

3D shape estimation in a constraint optimization neural network

One of the most important aspects of visual perception is the inference of 3D shape from a 2D retinal image of the outside world. The existence of several valid mapping functions from object to data makes this inverse problem ill-posed and therefore computationally difficult. In human vision, the retinal image is a 2D projection of the 3D world. The visual system imposes certain constraints on the family of solutions in order to uniquely and efficiently solve this inverse problem. This work specifically focused on the minimization of standard deviations of 3D angles (MSDA) for 3D perception. Our goal was to use a Deep Convolutional Neural Network based on biological principles derived from visual area V4 to achieve 3D reconstruction using constrained minimization of MSDA. We conducted an experiment using novel shapes with human subjects to collect data and test the model. The performance of the network largely agreed with how humans estimated novel 3D shapes. The results show that the constraint of MSDA in 3D shape can be implemented in a neural network and produce human-like results. Additional visual constraints can be added to the network in the future to fully test the theory of visual constraints as a basis of 3D shape perception.