Slant Perception Research Articles

On Human-like Biases in Convolutional Neural Networks for the Perception of Slant from Texture

Depth estimation is fundamental to 3D perception, and humans are known to have biased estimates of depth. This study investigates whether convolutional neural networks (CNNs) can be biased when predicting the sign of curvature and depth of surfaces of textured surfaces under different viewing conditions (field of view) and surface parameters (slant and texture irregularity). This hypothesis is drawn from the idea that texture gradients described by local neighborhoods—a cue identified in human vision literature—are also representable within convolutional neural networks. To this end, we trained both unsupervised and supervised CNN models on the renderings of slanted surfaces with random Polka dot patterns and analyzed their internal latent representations. The results show that the unsupervised models have similar prediction biases as humans across all experiments, while supervised CNN models do not exhibit similar biases. The latent spaces of the unsupervised models can be linearly separated into axes representing field of view and optical slant. For supervised models, this ability varies substantially with model architecture and the kind of supervision (continuous slant vs. sign of slant). Even though this study says nothing of any shared mechanism, these findings suggest that unsupervised CNN models can share similar predictions to the human visual system. Code: github.com/brownvc/Slant-CNN-Biases.

The contributions of surface features and contour shapes to object slant perception.

Humans perceive 3D shapes even from 2D images. A slant can be perceived from images of slanted rectangular objects, which include texture gradients and linear perspective contours. How does the visual system integrate and utilize these pictorial depth cues? A new visual illusion that provides some insights into this issue was examined. A box-like object with disk figures drawn on its upper surface was rendered in a linear perspective image. The length of the object's upper surface side line was overestimated, probably due to the foreshortened disks serving as slant cues. This illusory effect occurred even when observers estimated the line length on the image plane, suggesting that the slant perception from the disks was mandatory. Five experiments revealed that multiple depth cues were utilized for the slant perception; the aspect ratio of the disks, texture gradients, trapezium/parallelogram contours, and the side surfaces of the box-like object. However, foreshortened disks outside the object were not utilized as depth cues. These results suggested that various depth cues belonging to the target object are integrated for the slant perception.

The experience of stereoblindness does not improve use of texture for slant perception.

Stereopsis is an important depth cue for normal people, but a subset of people suffer from stereoblindness and cannot use binocular disparity as a cue to depth. Does this experience of stereoblindness modulate use of other depth cues? We investigated this question by comparing perception of 3D slant from texture for stereoblind people and stereo-normal people. Subjects performed slant discrimination and slant estimation tasks using both monocular and binocular stimuli. We found that two groups had comparable ability to discriminate slant from texture information and showed similar mappings between texture information and slant perception (biased perception toward frontal surface with texture information indicating low slants). The results suggest that the experience of stereoblindness did not change the use of texture information for slant perception. In addition, we found that stereoblind people benefitted from binocular viewing in the slant estimation task, despite their inability to use binocular disparity information. These findings are generally consistent with the optimal cue combination model of slant perception.

Investigation of optical texture properties as relative distance information for monocular guidance of reaching

Reaches guided using monocular versus binocular vision have been found to be equally fast and accurate only when optical texture was available projected from a support surface across which the reach was performed. We now investigate what property of optical texture elements is used to perceive relative distance: image width, image height, or image shape. Participants performed reaches to match target distances. Targets appeared on a textured surface on the left and participants reached to place their hand at target distance along a surface on the right. A perturbation discriminated which texture property was being used. The righthand surface was higher than the lefthand one by either 2, 4 or 6 cm. Participants should overshoot if they matched texture image width at the target, undershoot if they matched image shape, and undershoot far distances and, depending on the overall eye height, overshoot near distances if they matched image height. In Experiment 1, participants reached by moving a joystick to control a hand avatar in a virtual environment display. Their eye height was 15 cm. For each texture property, distances were predicted from the viewing geometry. Results ruled out image width in favor of image height or shape. In Experiment 2, participants at a 50 cm eye height reached in an actual environment with the same manipulations. Results supported use of image shape (or foreshortening), consistent with findings of texture properties used in slant perception. We discuss implications for models of visually guided reaching.

The perception of average slant is biased in concave surfaces

The perception of average slant is biased in concave surfaces

Slant perception becomes more biased with less reliable cues, even when stereo cues are available​

Slant perception becomes more biased with less reliable cues, even when stereo cues are available​

Multiple texture cues are integrated for perception of 3D slant from texture.

The projected image of a textured surface contains multiple texture cues to three-dimensional (3D) surface orientation. Previous studies have reported conflicting findings about the roles of various texture cues. We tested the influence of texture compression relative to other texture cues using a cue conflict paradigm. Observers viewed images of textured planar surfaces with varied slants (0°–70°) and estimated 3D slant by aligning their hand with the virtual surface. Conflicts between texture cues were created by stretching or compression the texture along the surface, which selectively changes the slant specified by texture compression. The texture distortions were relatively small (±10% or ±20%) to limit the size of the cue conflicts. Across three experiments, we varied the field of view (10° vs. 20°), texture regularity (circles vs. Voronoi), and availability of binocular cues. In monocular conditions, slant estimates were strongly affected by texture distortions. Analyses of cue weighting found that texture compression had more influence on slant settings than other texture cues and the relative influence of texture compression decreased with larger field of view and less regular textures. In binocular conditions, we also observed effects of texture distortion, and the influence of texture compression relative to information from stereo and other texture cues increased with slant. Our results provide evidence that texture compression contributes to perceived slant, in addition to other texture cues such as texture scaling. The observed effects of simulated slant, field of view, and texture regularity on cue weighting were all consistent with a model that integrates multiple sources of information according to their reliability.

A stratified process for the perception of objects: From optical transformations to 3D relief structure to 3D similarity structure to slant or aspect ratio

Previously, we developed a stratified process for slant perception. First, optical transformations in structure-from-motion (SFM) and stereo were used to derive 3D relief structure (where depth scaling remains arbitrary). Second, with sufficient continuous perspective change (≥45°), a bootstrap process derived 3D similarity structure. Third, the perceived slant was derived. As predicted by theoretical work on SFM, small visual angle (<5°) viewing requires non-coplanar points. Slanted surfaces with small 3D cuboids or tetrahedrons yielded accurate judgment while planar surfaces did not. Normally, object perception entails non-coplanar points. Now, we apply the stratified process to object perception where, after deriving similarity structure, alternative metric properties of the object can be derived (e.g. slant of the top surface or width-to-depth aspect ratio). First, we tested slant judgments of the smooth planar tops of three different polyhedral objects. We tested rectangular, hexagonal, and asymmetric pentagonal surfaces, finding that symmetry was required to determine the direction of slant (AP&P, 2019, https://doi.org/10.3758/s13414-019-01859-5). Our current results replicated the previous findings. Second, we tested judgments of aspect ratios, finding accurate performance only for symmetric objects. Results from this study suggest that, first, trackable non-coplanar points can be attained in the form of 3D objects. Second, symmetry is necessary to constrain slant and aspect ratio perception. Finally, deriving 3D similarity structure precedes estimating object properties, such as slant or aspect ratio. Together, evidence presented here supports the stratified bootstrap process for 3D object perception. Statement of significancePlanning interactions with objects in the surrounding environment entails the perception of 3D shape and slant. Studying ways through which 3D metric shape and slant can be perceived accurately by moving observers not only sheds light on how the visual system works, but also provides understanding that can be applied to other fields, like machine vision or remote sensing. The current study is a logical extension of previous studies by the same authors and explores the roles of large continuous perspective changes, relief structure, and symmetry in a stratified process for object perception.

Sensorimotor adaptation and cue reweighting compensate for distorted 3D shape information, accounting for paradoxical perception-action dissociations.

Visually guided movements can show surprising accuracy even when the perceived three-dimensional (3D) shape of the target is distorted. One explanation of this paradox is that an evolutionarily specialized "vision-for-action" system provides accurate shape estimates by relying selectively on stereo information and ignoring less reliable sources of shape information like texture and shading. However, the key support for this hypothesis has come from studies that analyze average behavior across many visuomotor interactions where available sensory feedback reinforces stereo information. The present study, which carefully accounts for the effects of feedback, shows that visuomotor interactions with slanted surfaces are actually planned using the same cue-combination function as slant perception and that apparent dissociations can arise due to two distinct supervised learning processes: sensorimotor adaptation and cue reweighting. In two experiments, we show that when a distorted slant cue biases perception (e.g., surfaces appear flattened by a fixed amount), sensorimotor adaptation rapidly adjusts the planned grip orientation to compensate for this constant error. However, when the distorted slant cue is unreliable, leading to variable errors across a set of objects (i.e., some slants are overestimated, others underestimated), then relative cue weights are gradually adjusted to reduce the misleading effect of the unreliable cue, consistent with previous perceptual studies of cue reweighting. The speed and flexibility of these two forms of learning provide an alternative explanation of why perception and action are sometimes found to be dissociated in experiments where some 3D shape cues are consistent with sensory feedback while others are faulty.NEW & NOTEWORTHY When interacting with three-dimensional (3D) objects, sensory feedback is available that could improve future performance via supervised learning. Here we confirm that natural visuomotor interactions lead to sensorimotor adaptation and cue reweighting, two distinct learning processes uniquely suited to resolve errors caused by biased and noisy 3D shape cues. These findings explain why perception and action are often found to be dissociated in experiments where some cues are consistent with sensory feedback while others are faulty.

The Impact of Radial Distortions in VR Headsets on Perceived Surface Slant

Abstract Modern virtual reality (VR) headsets use lenses that distort the visual field, typically with distortion increasing with eccentricity. While content is pre-warped to counter this radial distortion, residual image distortions remain. Here we examine the extent to which such residual distortion impacts the perception of surface slant. In Experiment 1, we presented slanted surfaces in a head-mounted display and observers estimated the local surface slant at different locations. In Experiments 2 (slant estimation) and 3 (slant discrimination), we presented stimuli on a mirror stereoscope, which allowed us to more precisely control viewing and distortion parameters. Taken together, our results show that radial distortion has significant impact on perceived surface attitude, even following correction. Of the distortion levels we tested, 5% distortion results in significantly underestimated and less precise slant estimates relative to distortion-free surfaces. In contrast, Experiment 3 reveals that a level of 1% distortion is insufficient to produce significant changes in slant perception. Our results highlight the importance of adequately modeling and correcting lens distortion to improve VR user experience.

Bootstrapping a better slant: A stratified process for recovering 3D metric slant.

Lind et al. (Journal of Experimental Psychology: Human Perception and Performance, 40 (1), 83, 2014) proposed a bootstrap process that used right angles on 3D relief structure, viewed over sufficiently large continuous perspective change, to recover the scaling factor for metric shape. Wang, Lind, and Bingham (Journal of Experimental Psychology: Human Perception and Performance, 44(10), 1508-1522, 2018) replicated these results in the case of 3D slant perception. However, subsequent work by the same authors (Wang et al., 2019) suggested that the original solution could be ineffective for 3D slant and presented an alternative that used two equidistant points (a portion of the original right angle). We now describe a three-step stratified process to recover 3D slant using this new solution. Starting with 2D inputs, we (1) used an existing structure-from-motion (SFM) algorithm to derive the object's 3D relief structure and (2) applied the bootstrap process to it to recover the unknown scaling factor, which (3) was then used to produce a slant estimate. We presented simulations of results from four previous experiments (Wang et al., 2018, 2019) to compare model and human performance. We showed that the stratified process has great predictive power, reproducing a surprising number of phenomena found in human experiments. The modeling results also confirmed arguments made in Wang et al. (2019) that an axis of mirror symmetry in an object allows observers to use the recovered scaling factor to produce an accurate slant estimate. Thus, poor estimates in the context of a lack of symmetry do not mean that the scaling factor has not been recovered, but merely that the direction of slant was ambiguous.

Symmetry mediates the bootstrapping of 3-D relief slant to metric slant.

Empirical studies have always shown 3-D slant and shape perception to be inaccurate as a result of relief scaling (an unknown scaling along the depth direction). Wang, Lind, and Bingham (Journal of Experimental Psychology: Human Perception and Performance, 44(10), 1508-1522, 2018) discovered that sufficient relative motion between the observer and 3-D objects in the form of continuous perspective change (≥45°) could enable accurate 3-D slant perception. They attributed this to a bootstrap process (Lind, Lee, Mazanowski, Kountouriotis, & Bingham in Journal of Experimental Psychology: Human Perception and Performance, 40(1), 83, 2014) where the perceiver identifies right angles formed by texture elements and tracks them in the 3-D relief structure through rotation to extrapolate the unknown scaling factor, then used to convert 3-D relief structure to 3-D Euclidean structure. This study examined the nature of the bootstrap process in slant perception. In a series of four experiments, we demonstrated that (1) features of 3-D relief structure, instead of 2-D texture elements, were tracked (Experiment 1); (2) identifying right angles was not necessary, and a different implementation of the bootstrap process is more suitable for 3-D slant perception (Experiment 2); and (3) mirror symmetry is necessary to produce accurate slant estimation using the bootstrapped scaling factor (Experiments 3 and 4). Together, the results support the hypothesis that a symmetry axis is used to determine the direction of slant and that 3-D relief structure is tracked over sufficiently large perspective change to produce metric depth. Altogether, the results supported the bootstrap process.

Slant perception in the presence of curvature distortion

Slant perception in the presence of curvature distortion

Does experience of stereoblindness change use of texture cues in slant perception?

Does experience of stereoblindness change use of texture cues in slant perception?

Perception of 3D slant from textures with and without aligned spectral components

Perception of 3D slant from textures with and without aligned spectral components

Perception of 3D slant from textures with and without aligned spectral components.

The image of a textured surface provides multiple potential cues that could be used to perceive its 3D structure. When a surface texture has oriented structure, perspective convergence of the oriented components provides a texture cue, in addition to texture scaling and compression. Some findings suggest that oriented spectral components aligned with the direction of slant are important for 3D perception from texture, but this has only been demonstrated in restricted situations. In this study, we tested the contribution of oriented spectral components for planar surface patches with varied slant (0°-80°) and field of view (16° and 6°). Observers viewed simulated textured surfaces and estimated slant to the perceived surfaces. The simulated textures were octotropic plaids with a full range of orientations, or with either the aligned or perpendicular plaid components removed. We measured perceptual bias in monocular conditions and also the relative weighting of texture and stereo information in binocular conditions. We found that the presence of aligned spectral components did produce some improvement in slant estimates, but the effects were small and only observed in some conditions. There were no detectable differences in texture cue weights. Our results demonstrate that aligned spectral components contribute to perception of slant from texture, but suggest that the contribution is limited when other texture cues are available and informative. Our findings are consistent with the notion that the visual system utilizes multiple sources of texture information for 3D perception.

Pedal and haptic estimates of slant suggest a common underlying representation

It is well known that people verbally exaggerate the slant of visually perceived geographical, virtual, and man-made hills. More recently it has been shown that haptic and verbal estimates of slant result in similar exaggerations, supporting the proprioception calibration hypothesis—that similar biases exist in both verbal estimates of visually perceived slant and proprioceptively perceived hand orientation. This seems to point to a common underlying representation of slant.However, it is unclear if and how manual proprioceptive estimates might be relevant for perception of ground surface slant or how this might translate to pedal perception of surface orientation. In the current work we tested whether pedal perception is systematically connected to a representational system shared by haptic and visual perception. We did this by having people orient their foot to four different orientations of a ramp (Experiment 1) or to a staircase (Experiment 2) and compared these to estimates made using a free hand measure as well as to verbal estimates. Our results show that verbal, haptic, and pedal measures of visually perceived surface orientation all result in similar estimates of slant and do so across different slanted surfaces. This suggests that verbal and haptic proprioceptive estimates tap into a representational system of visually perceived surface orientation that is relevant for walking up various surface orientations.

Large continuous perspective change with noncoplanar points enables accurate slant perception.

Perceived slant has often been characterized as a component of 3D shape perception for polyhedral objects. Like 3D shape, slant is often perceived inaccurately. Lind, Lee, Mazanowski, Kountouriotis, and Bingham (2014) found that 3D shape was perceived accurately with perspective changes ≥ 45°. We now similarly tested perception of 3D slant. To account for their results, Lind et al. (2014) developed a bootstrap model based on the assumption that optical information yields perception of 3D relief structure then used with large perspective changes to bootstrap to perception of 3D Euclidean structure. However, slant perception usually entails planar surfaces and structure-from-motion fails in the absence of noncoplanar points. Nevertheless, the displays in Lind et al. (2014) included stereomotion in addition to monocular optical flow. Because stereomotion is higher order, the bootstrap model might apply in the case of strictly planar surfaces. We investigated whether stereomotion, monocular structure-from-motion (SFM), or the combination of the two would yield accurate 3D slant perception with large continuous perspective change. In Experiment 1, we found that judgments of slant were inaccurate in all information conditions. In Experiment 2, we added noncoplanar structure to the surfaces. We found that judgments in the monocular SFM and combined conditions now became correct once perspective changes were ≥ 45°, replicating the results of Lind et al. (2014) and supporting the bootstrap model. In short, we found that noncoplanar structure was required to enable accurate perception of 3D slant with sufficiently large perspective changes. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Searching for Invariance: Geographical and Optical Slant

When we move through rigid environments, surface orientations of static objects do not appear to change. Most studies have investigated the perception of optical slant which is dependent on the perspective of the observer. We investigated the perception of geographical slant, which is invariant across different viewing perspectives, and compared it to optical slant. In Experiment 1, participants viewed a 3D triangular target surface with triangular phosphorescent texture elements presented at eye level at one of 5 slants from 0° to 90°, at 0° or 40° tilt. Participants turned around to adjust a 2D line or a 3D surface to match the slant of the target surface. In Experiment 2, the difference between optical and geographical slant was increased by changing the height of the surface to be judged. In Experiment 3, target surfaces were rotated by 50° (±25°) and viewed in both a dark and lighted room. In Experiment 1, the overall pattern of judgments exhibited only slight differences between response measures. In Experiment 2, slant judgments were slightly overestimated when the surface was at a low height and at 0° tilt. We compared optical slants of the surfaces to geographical slants. While sometimes inaccurate, participants' slant judgments remained invariant across changes in viewing perspective. In Experiment 3, judgments were the same in the dark and lighted conditions. There was no effect of target motion on judgments, although variability decreased. We conclude that participants' judgments were predicted by geographical slant, not optical slant.

Texture compression and scaling both contribute to perception of 3D slant from texture

Texture compression and scaling both contribute to perception of 3D slant from texture