3D Shape Perception Research Articles

A stratified process for the perception of objects: From optical transformations to 3D relief structure to 3D similarity structure to slant or aspect ratio

Previously, we developed a stratified process for slant perception. First, optical transformations in structure-from-motion (SFM) and stereo were used to derive 3D relief structure (where depth scaling remains arbitrary). Second, with sufficient continuous perspective change (≥45°), a bootstrap process derived 3D similarity structure. Third, the perceived slant was derived. As predicted by theoretical work on SFM, small visual angle (<5°) viewing requires non-coplanar points. Slanted surfaces with small 3D cuboids or tetrahedrons yielded accurate judgment while planar surfaces did not. Normally, object perception entails non-coplanar points. Now, we apply the stratified process to object perception where, after deriving similarity structure, alternative metric properties of the object can be derived (e.g. slant of the top surface or width-to-depth aspect ratio). First, we tested slant judgments of the smooth planar tops of three different polyhedral objects. We tested rectangular, hexagonal, and asymmetric pentagonal surfaces, finding that symmetry was required to determine the direction of slant (AP&P, 2019, https://doi.org/10.3758/s13414-019-01859-5). Our current results replicated the previous findings. Second, we tested judgments of aspect ratios, finding accurate performance only for symmetric objects. Results from this study suggest that, first, trackable non-coplanar points can be attained in the form of 3D objects. Second, symmetry is necessary to constrain slant and aspect ratio perception. Finally, deriving 3D similarity structure precedes estimating object properties, such as slant or aspect ratio. Together, evidence presented here supports the stratified bootstrap process for 3D object perception. Statement of significancePlanning interactions with objects in the surrounding environment entails the perception of 3D shape and slant. Studying ways through which 3D metric shape and slant can be perceived accurately by moving observers not only sheds light on how the visual system works, but also provides understanding that can be applied to other fields, like machine vision or remote sensing. The current study is a logical extension of previous studies by the same authors and explores the roles of large continuous perspective changes, relief structure, and symmetry in a stratified process for object perception.

The Veiled Virgin illustrates visual segmentation of shape by cause

Three-dimensional (3D) shape perception is one of the most important functions of vision. It is crucial for many tasks, from object recognition to tool use, and yet how the brain represents shape remains poorly understood. Most theories focus on purely geometrical computations (e.g., estimating depths, curvatures, symmetries). Here, however, we find that shape perception also involves sophisticated inferences that parse shapes into features with distinct causal origins. Inspired by marble sculptures such as Strazza's The Veiled Virgin (1850), which vividly depict figures swathed in cloth, we created composite shapes by wrapping unfamiliar forms in textile, so that the observable surface relief was the result of complex interactions between the underlying object and overlying fabric. Making sense of such structures requires segmenting the shape based on their causes, to distinguish whether lumps and ridges are due to the shrouded object or to the ripples and folds of the overlying cloth. Three-dimensional scans of the objects with and without the textile provided ground-truth measures of the true physical surface reliefs, against which observers' judgments could be compared. In a virtual painting task, participants indicated which surface ridges appeared to be caused by the hidden object and which were due to the drapery. In another experiment, participants indicated the perceived depth profile of both surface layers. Their responses reveal that they can robustly distinguish features belonging to the textile from those due to the underlying object. Together, these findings reveal the operation of visual shape-segmentation processes that parse shapes based on their causal origin.

Hole-in-the-wall: Perception of 3D shape and affordances from static images in humans and machines

Hole-in-the-wall: Perception of 3D shape and affordances from static images in humans and machines

The Intrinsic Constraint Model: A non-Euclidean approach to 3D shape perception from multiple image signals

The Intrinsic Constraint Model: A non-Euclidean approach to 3D shape perception from multiple image signals

Deconstructing the relief inversion effect: Contributors of the problem and its solutions

Deconstructing the relief inversion effect: Contributors of the problem and its solutions

Computational Model for Human 3D Shape Perception From a Single Specular Image.

In natural conditions the human visual system can estimate the 3D shape of specular objects even from a single image. Although previous studies suggested that the orientation field plays a key role for 3D shape perception from specular reflections, its computational plausibility, and possible mechanisms have not been investigated. In this study, to complement the orientation field information, we first add prior knowledge that objects are illuminated from above and utilize the vertical polarity of the intensity gradient. Then we construct an algorithm that incorporates these two image cues to estimate 3D shapes from a single specular image. We evaluated the algorithm with glossy and mirrored surfaces and found that 3D shapes can be recovered with a high correlation coefficient of around 0.8 with true surface shapes. Moreover, under a specific condition, the algorithm's errors resembled those made by human observers. These findings show that the combination of the orientation field and the vertical polarity of the intensity gradient is computationally sufficient and probably reproduces essential representations used in human shape perception from specular reflections.

Bodily awareness and novel multisensory features

According to the decomposition thesis, perceptual experiences resolve without remainder into their different modality-specific components. Contrary to this view, I argue that certain cases of multisensory integration give rise to experiences representing features of a novel type. Through the coordinated use of bodily awareness—understood here as encompassing both proprioception and kinaesthesis—and the exteroceptive sensory modalities, one becomes perceptually responsive to spatial features whose instances couldn’t be represented by any of the contributing modalities functioning in isolation. I develop an argument for this conclusion focusing on two cases: 3D shape perception in haptic touch and experiencing an object’s egocentric location in crossmodally accessible, environmental space.

Photogeometric Cues to Perceived Surface Shading

The human visual system is remarkably adept at extracting the three-dimensional (3D) shape of surfaces from images of smoothly shaded surfaces (shape from shading). Most research into this remarkable perceptual ability has focused on understanding how the visual system derives a specific representation of 3D shape when it is known (or assumed) that shading and self-occluding contours are the sole causes of image structure [1-11]. But there is an even more fundamental problem that must be solved before any such analysis can take place: how does the visual system determine when it's viewing a shaded surface? Here, we present theoretical analyses showing that there is statistically reliable information generated along the bounding contours of smoothly curved surfaces that the visual system uses to identify surface shading. This information can be captured by two photogeometric constraints that link the shape of bounding contours to the distributions of shading intensity along the contours: one that links shading intensity to the local orientations along bounding contours and a second that links shading intensity to bounding contour curvature. We show that these constraints predict the perception of shading for surfaces with smooth self-occluding contours and a widely studied class of bounding contours (planar cuts). The results provide new insights into the information that the visual system exploits to distinguish surface shading from other sources of image structure and offer a coherent explanation of the influence of bounding contours on the perception of surface shading and 3D shape.

Large continuous perspective change with noncoplanar points enables accurate slant perception.

Perceived slant has often been characterized as a component of 3D shape perception for polyhedral objects. Like 3D shape, slant is often perceived inaccurately. Lind, Lee, Mazanowski, Kountouriotis, and Bingham (2014) found that 3D shape was perceived accurately with perspective changes ≥ 45°. We now similarly tested perception of 3D slant. To account for their results, Lind et al. (2014) developed a bootstrap model based on the assumption that optical information yields perception of 3D relief structure then used with large perspective changes to bootstrap to perception of 3D Euclidean structure. However, slant perception usually entails planar surfaces and structure-from-motion fails in the absence of noncoplanar points. Nevertheless, the displays in Lind et al. (2014) included stereomotion in addition to monocular optical flow. Because stereomotion is higher order, the bootstrap model might apply in the case of strictly planar surfaces. We investigated whether stereomotion, monocular structure-from-motion (SFM), or the combination of the two would yield accurate 3D slant perception with large continuous perspective change. In Experiment 1, we found that judgments of slant were inaccurate in all information conditions. In Experiment 2, we added noncoplanar structure to the surfaces. We found that judgments in the monocular SFM and combined conditions now became correct once perspective changes were ≥ 45°, replicating the results of Lind et al. (2014) and supporting the bootstrap model. In short, we found that noncoplanar structure was required to enable accurate perception of 3D slant with sufficiently large perspective changes. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

A machine learning framework for full-reference 3D shape quality assessment

To decide whether the perceived quality of a mesh is influenced by a certain modification such as compression or simplification, a metric for estimating the visual quality of 3D meshes is required. Today, machine learning and deep learning techniques are getting increasingly popular since they present efficient solutions to many complex problems. However, these techniques are not much utilized in the field of 3D shape perception. We propose a novel machine learning-based approach for evaluating the visual quality of 3D static meshes. The novelty of our study lies in incorporating crowdsourcing in a machine learning framework for visual quality evaluation. We deliberate that this is an elegant way since modeling human visual system processes is a tedious task and requires tuning many parameters. We employ crowdsourcing methodology for collecting data of quality evaluations and metric learning for drawing the best parameters that well correlate with the human perception. Experimental validation of the proposed metric reveals a promising correlation between the metric output and human perception. Results of our crowdsourcing experiments are publicly available for the community.

Comparing the terrain reversal effect in satellite images and in shaded relief maps: an examination of the effects of color and texture on 3D shape perception from shading

ABSTRACTTerrain reversal effect (TRE) causes reversed 3D shape perception in satellite images and shaded relief maps (SRMs), and introduces difficulties in identifying landforms such as valleys and ridges. With this paper, in a controlled laboratory experiment, we compare how well 27 participants could identify valleys and ridges over 33 locations using SRMs, color satellite images and grayscale satellite images. The main depth cue is shadow both in vertical-view images and SRMs. However, the presence of texture and color in images also affect 3D shape perception. All our participants experience the illusion strongly: with the SRMs, it is very severe (2% accuracy), with grayscale images low but considerably better than SRMs (17.6% accuracy), and slightly worse with color imagery (15.3% accuracy). These differences between SRMs and imagery suggest that the participants who are able to bypass the illusion consciously or subconsciously interpret the photographic information. We support this observation further with a cue-strength analysis. Furthermore, we provide exploratory analyses of the effects of expertise, global convexity bias, and bistable perception. Our original empirical observations serve towards a better understanding of this visual illusion, and contribute towards nuanced and appropriate solutions to correcting for TRE differently for satellite images and SRMs.

Binocular stereo acuity affects monocular three-dimensional shape perception in patients with strabismus

Background/aimsTo evaluate the perception of three-dimensional (3D) shape in patients with strabismus and the contributions of stereopsis and monocular cues to this perception.MethodsTwenty-one patients with strabismus with and 20 without...

Effects of stereoscopic disparity on early ERP components during classification of three-dimensional objects.

This study investigates the effects of stereo disparity on the perception of three-dimensional (3D) object shape. We tested the hypothesis that stereo input modulates the brain activity related to perceptual analyses of 3D shape configuration during image classification. High-density (256-channel) electroencephalogram (EEG) was used to record the temporal dynamics of visual shape processing under conditions of two-dimensional (2D) and 3D visual presentation. On each trial, observers made image classification judgements ('Same'/'Different') to two briefly presented, multi-part, novel objects. On different-object trials, stimuli could either share volumetric parts but not the global 3D shape configuration and have different parts but the same global 3D shape configuration or differ on both aspects. Analyses using mass univariate contrasts showed that the earliest sensitivity to 2D versus 3D viewing appeared as a negative deflection over posterior locations on the N1 component between 160 and 220 ms post-stimulus onset. Subsequently, event-related potential (ERP) modulations during the N2 time window between 240 and 370 ms were linked to image classification. N2 activity reflected two distinct components - an early N2 (240-290 ms) and a late N2 (290-370 ms) - that showed different patterns of responses to 2D and 3D input and differential sensitivity to 3D object structure. The results revealed that stereo input modulates the neural correlates of 3D object shape. We suggest that this reflects differential perceptual processing of object shape under conditions of stereo or mono input. These findings challenge current theories that attribute no functional role for stereo input during 3D shape perception.

Improving 3D Shape Recognition withElectrostatic Friction Display.

Electrovibration technology has the potential for seamless integration into ordinary smartphones and tablets to provide programmable haptic feedback. The aim of this work is to seek effective ways to improve 3D perception of visual objects rendered on an electrovibration display. Utilizing a gradient-based algorithm, we first investigated whether rendering only lateral frictional force on an electrovibration display improves 3D shape perception compared to doing the same using a force-feedback interface. We observed that although users do not naturally associate electrovibration patterns to geometrical shapes, they can map patterns to shapes with moderate accuracy if guidance or context is given. Motivated by this finding, we generalized the gradient-based rendering algorithm to estimate the surface gradient for any 3D mesh and added an edge detection algorithm to render sharp edges. Then, we evaluated the advantages of our algorithm in a user study and found that our algorithm can notably improve the performance of 3D shape recognition when visual information is limited.

Binocular 3D Object Recovery Using a Symmetry Prior

We present a new algorithm for 3D shape reconstruction from stereo image pairs that uses mirror symmetry as a biologically inspired prior. 3D reconstruction requires some form of prior because it is an ill-posed inverse problem. Psychophysical research shows that mirror-symmetry is a key prior for 3D shape perception in humans, suggesting that a general purpose solution to this problem will have many applications. An approach is developed for finding objects that fit a given shape definition. The algorithm is developed for shapes with two orthogonal planes of symmetry, thus allowing for straightforward recovery of occluded portions of the objects. Two simulations were run to test: (1) the accuracy of 3D recovery, and (2) the ability of the algorithm to find the object in the presence of noise. We then tested the algorithm on the Children’s Furniture Corpus, a corpus of stereo image pairs of mirror symmetric furniture objects. Runtimes and 3D reconstruction errors are reported and failure modes described.

Local-to-global mesh saliency

As a measure of regional importance in agreement with human perception of 3D shape, mesh saliency should be based on local geometric information within a mesh but more than that. Recent research has shown that global consideration has a significant role in mesh saliency. This paper proposes a local-to-global framework for computing mesh saliency where we offer novel solutions to solve three inherent problems: (1) an algorithm based on statistic Laplacian which does not only compute local saliency, but also facilitates the later computation of global saliency; (2) a local-to-global method based on pooling and global distinctness to compute global saliency; (3) a framework to integrate local and global saliency. Experiments demonstrate that our approach can effectively detect salient features consistent with human perceptual interest. We also provide comparisons to existing state-of-the-art methods for mesh saliency and show improved results produced by our method.

The effects of illumination on the perception of 3D shape from shading

The effects of illumination on the perception of 3D shape from shading

The perception of 3D shape from shading based entirely on transmitted light

The perception of 3D shape from shading based entirely on transmitted light

3D-SHAPE PERCEPTION IN AMNESTIC MILD COGNITIVE IMPAIRMENT

3D-SHAPE PERCEPTION IN AMNESTIC MILD COGNITIVE IMPAIRMENT

The darker-is-deeper heuristic for the perception of 3D shape from shading: Is it perceptually or ecologically valid?

The darker-is-deeper heuristic was originally proposed by Langer and Zucker (1994) for approximating 3D shape from shading under conditions of diffuse illumination that typically occur for outdoor scenes on a cloudy day, and it is based on the assumption that vignetting is the primary source of luminance variation under those conditions. It was later rejected as a model of human perception by Langer and Bülthoff (2000), because points in concavities that appear to be the deepest are most often located on local luminance maxima. Despite that result, this heuristic has continued to be described in the literature as a viable model of human perception (e.g., Chen & Tyler, 2015; Tyler, 1998), based entirely on the appearance of image intensity gratings, which have little or no connection to real 3D surfaces or patterns of illumination. In this article we will present a large number of examples to show what actually happens when surfaces are viewed under directional and diffuse illuminations. The results will highlight a number of well-known phenomena in addition to vignetting that can influence the pattern of shading on a surface under diffuse illumination, and they will also demonstrate that the darker-is-deeper heuristic is generally invalid for all types of illumination, except in unusual circumstances.