Depth Plane Research Articles

The accuracy of distance perception in the frontal plane of projection‐based stereoscopic environments

AbstractAn experiment was conducted to investigate the accuracy of distance judgment in a frontal plane of projection‐based stereoscopic environments. The targets were presented at nine distinct frontal plane positions and at three depth levels. Eighteen right‐handed participants with self‐declared normal visual acuity reached the target, either continuously visible or presented briefly, by holding pointing sticks. All combinations of the experimental conditions were repeated for an equivalent real‐world environment. Accuracies of judgments were then computed from three‐dimensional data collected by a motion system composed of six infrared cameras. As compared with about 94% accuracy in the physical world, the space in the frontal plane was only about 85% on the stereoscopic environment. The result also revealed more accurate judgment in the continuous presence of the target. Furthermore, the accuracy was affected by the egocentric distance of the frontal plane from the position of the participant. The study concluded that the compression in the frontal plane and underestimation of depth, which has been reported by the majority of previous studies could be an indication that the space compression in virtual environments might be in all the three‐dimensions. These findings can be used as guidelines for developers and content designers to properly locate virtual targets and selection of efficient interaction modes.

Effect of apparent depth cues on accommodation in a Badal optometer

BackgroundThe aim was to analyse the effect of peripheral depth cues on accommodation in Badal optometers.MethodsMonocular refractions at 0.17 and 5.00 D of accommodative stimulus were measured with the PowerRef II autorefractor (Plusoptix Inc., Atlanta, Georgia, USA). Subjects looked (randomly) at four different scenes, one real scene comprising familiar objects at different depth planes (Real) and three virtual scenes comprising different two‐dimensional pictures seen through a Badal lens. The first image consisted of a photograph of the real scene taken in conditions that closely mimic a healthy standard human eye performance (out‐of‐focus [OoF] blur); the second image was the same photograph rendered with a depth of focus to infinity (OoF sharpness); and finally the third image consisted of a fixation target and a even white surrounding (White). In all cases the field of view was 25.0° and the fixation target was a Maltese cross subtending to two degrees.ResultsTwenty‐eight right eyes from healthy young subjects were measured. The achieved statistical power was 0.9. At 5.00 D of accommodative stimulus, the repeated measures analysis of variance was statistically significant (p < 0.05) and the corresponding Bonferroni post hoc tests showed the following mean accommodative response differences and standard deviation (p‐value) between the real and the virtual scenes: real‐white =−0.66 ± 0.92 D (p < 0.01); real‐OoF sharpness = −0.43 ± 0.88 D (p = 0.07); real‐OoF blur =−0.25 ± 0.93 D (p = 0.89).ConclusionsA stimulus poor in depth cues inaccurately stimulates accommodation in Badal optometers; however, accommodation can be significantly improved in the same Badal optometer, when displaying a realistic image rich in peripheral depth cues, even though these peripheral cues (also referred to as retinal blur cues) are shown in the same plane as the fixation target. These results have important implications in stereoscopic virtual reality systems that fail to represent appropriately retinal blur.

Vergence driven accommodation with simulated disparity in myopia and emmetropia

The formation of focused and corresponding foveal images requires a close synergy between the accommodation and vergence systems. This linkage is usually decoupled in virtual reality systems and may be dysfunctional in people who are at risk of developing myopia. We study how refractive error affects vergence-accommodation interactions in stereoscopic displays. Vergence and accommodative responses were measured in 21 young healthy adults (n=9 myopes, 22–31 years) while subjects viewed naturalistic stimuli on a 3D display. In Step 1, vergence was driven behind the monitor using a blurred, non-accommodative, uncrossed disparity target. In Step 2, vergence and accommodation were driven back to the monitor plane using naturalistic images that contained structured depth and focus information from size, blur and/or disparity. In Step 1, both refractive groups converged towards the stereoscopic target depth plane, but the vergence-driven accommodative change was smaller in emmetropes than in myopes (F1,19=5.13, p=0.036). In Step 2, there was little effect of peripheral depth cues on accommodation or vergence in either refractive group. However, vergence responses were significantly slower (F1,19=4.55, p=0.046) and accommodation variability was higher (F1,19=12.9, p=0.0019) in myopes. Vergence and accommodation responses are disrupted in virtual reality displays in both refractive groups. Accommodation responses are less stable in myopes, perhaps due to a lower sensitivity to dioptric blur. Such inaccuracies of accommodation may cause long-term blur on the retina, which has been associated with a failure of emmetropization.

Light Field Compression With Homography-Based Low-Rank Approximation

This paper describes a light field compression scheme based on a novel homography-based low-rank approximation method called HLRA. The HLRA method jointly searches for the set of homographies best aligning the light field views and for the low-rank approximation matrices. The light field views are aligned using either one global homography or multiple homographies depending on how much the disparity across views varies from one depth plane to the other. The light field low-rank representation is then compressed using high efficiency video coding (HEVC). The best pair of rank and quantization parameters of the coding scheme, for a given target bit rate, is predicted with a model defined as a function of light field disparity and texture features. The results are compared with those obtained by directly applying HEVC on the light field views restructured as a pseudovideo sequence. The experiments using different datasets show substantial peak signal to noise ratio (PSNR)-rate gain of our compression algorithm, as well as the accuracy of the proposed parameter prediction model, especially for real light fields. A scalable extension of the coding scheme is finally proposed.

Adding depth to overlapping displays can improve visual search performance.

Standard models of visual search have focused upon asking participants to search for a single target in displays where the objects do not overlap one another, and where the objects are presented on a single depth plane. This stands in contrast to many everyday visual searches wherein variations in overlap and depth are the norm, rather than the exception. Here, we addressed whether presenting overlapping objects on different depths planes to one another can improve search performance. Across 4 different experiments using different stimulus types (opaque polygons, transparent polygons, opaque real-world objects, and transparent X-ray images), we found that depth was primarily beneficial when the displays were transparent, and this benefit arose in terms of an increase in response accuracy. Although the benefit to search performance only appeared in some cases, across all stimulus types, we found evidence of marked shifts in eye-movement behavior. Our results have important implications for current models and theories of visual search, which have not yet provided detailed accounts of the effects that overlap and depth have on guidance and object identification processes. Moreover, our results show that the presence of depth information could aid real-world searches of complex, overlapping displays. (PsycINFO Database Record

Transparency in stereopsis: parallel encoding of overlapping depth planes.

We report that after extensive training, expert adults can accurately report the number, up to six, of transparent overlapping depth planes portrayed by brief (400ms or 200ms) random-element stereoscopic displays, and can well discriminate six from seven planes. Naïve subjects did poorly above three planes. Displays contained seven rows of 12 randomly located ×'s or +'s; jittering the disparities and number in each row to remove spurious cues had little effect on accuracy. Removing the central 3° of the 10° display to eliminate foveal vision hardly reduced the number of reportable planes. Experts could report how many of six planes contained +'s when the remainder contained ×'s, and most learned to report up to six planes in reverse contrast (left eye white +'s; right eye black +'s). Long-term training allowed some experts to reach eight depth planes. Results suggest that adult stereoscopic vision can learn to distinguish the outputs of six or more statistically independent, contrast-insensitive, narrowly tuned, asymmetric disparity channels in parallel.

Evidence for the effect of depth on visual working memory

Visual working memory (VWM) is a cognitive memory buffer for temporarily holding, processing, and manipulating visual information. Previous studies have demonstrated mixed results of the effect of depth perception on VWM, with some showing a beneficial effect while others not. In this study, we employed an adapted change detection paradigm to investigate the effects of two depth cues, binocular disparity and relative size. The memory array consisted of a set of pseudo-randomly positioned colored items, and the task was to judge whether the test item was changed compared to the memory item after a retention interval. We found that presenting the items in stereoscopic depth alone hardly affected VWM performance. When combining the two coherent depth cues, a significant larger VWM capacity of the perceptually closer-in-depth items was observed than that of the farther items, but the capacity for the two-depth-planes condition was not significantly different from that for the one-plane condition. Conflicting the two depth cues resulted in cancelling the beneficial effect of presenting items at a closer depth plane. The results indicate that depth perception could affect VWM, and the visual system may have an advantage in maintaining closer-in-depth objects in working memory.

Short-term visual memory for location in depth: A U-shaped function of time.

Short-term visual memory was studied by displaying arrays of four or five numerals, each numeral in its own depth plane, followed after various delays by an arrow cue shown in one of the depth planes. Subjects reported the numeral at the depth cued by the arrow. Accuracy fell with increasing cue delay for the first 500 ms or so, and then recovered almost fully. This dipping pattern contrasts with the usual iconic decay observed for memory traces. The dip occurred with or without a verbal or color-shape retention load on working memory. In contrast, accuracy did not change with delay when a tonal cue replaced the arrow cue. We hypothesized that information concerning the depths of the numerals decays over time in sensory memory, but that cued recall is aided later on by transfer to a visual memory specialized for depth. This transfer is sufficiently rapid with a tonal cue to compensate for the sensory decay, but it is slowed by the need to tag the arrow cue's depth relative to the depths of the numerals, exposing a dip when sensation has decayed and transfer is not yet complete. A model with a fixed rate of sensory decay and varied transfer rates across individuals captures the dip as well as the cue modality effect.

Contextual cueing in 3D visual search depends on representations in planar-, not depth-defined space.

Learning of spatial inter-item associations can speed up visual search in everyday life, an effect referred to as contextual cueing (Chun & Jiang, 1998). Whereas previous studies investigated contextual cueing primarily using 2D layouts, the current study examined how 3D depth influences contextual learning in visual search. In two experiments, the search items were presented evenly distributed across front and back planes in an initial training session. In the subsequent test session, the search items were either swapped between the front and back planes (Experiment 1) or between the left and right halves (Experiment 2) of the displays. The results showed that repeated spatial contexts were learned efficiently under 3D viewing conditions, facilitating search in the training sessions, in both experiments. Importantly, contextual cueing remained robust and virtually unaffected following the swap of depth planes in Experiment 1, but it was substantially reduced (to nonsignificant levels) following the left-right side swap in Experiment 2. This result pattern indicates that spatial, but not depth, inter-item variations limit effective contextual guidance. Restated, contextual cueing (even under 3D viewing conditions) is primarily based on 2D inter-item associations, while depth-defined spatial regularities are probably not encoded during contextual learning. Hence, changing the depth relations does not impact the cueing effect.

Visual selective attention with virtual barriers.

Previous studies have shown that interference effects in the flanker task are reduced when physical barriers (e.g., hands) are placed around rather than below a target flanked by distractors. One explanation of this finding is the referential coding hypothesis, whereby the barriers serve as reference objects for allocating attention. In five experiments, the generality of the referential coding hypothesis was tested by investigating whether interference effects are modulated by the placement of virtual barriers (e.g., parentheses). Modulation of flanker interference was found only when target and distractors differed in size and the virtual barriers were beveled wood-grain objects. Under these conditions and those of previous studies, the author conjectures that an impression of depth was produced when the barriers were around the target, such that the target was perceived to be on a different depth plane than the distractors. Perception of depth in the stimulus display might have led to referential coding of the stimuli in three-dimensional (3-D) space, influencing the allocation of attention beyond the horizontal and vertical dimensions. This 3-D referential coding hypothesis is consistent with research on selective attention in 3-D space that shows flanker interference is reduced when target and distractors are separated in depth.

Stereoscopic Segmentation Cues Improve Visual Timing Performance in Spatiotemporally Cluttered Environments.

Recently, Cass and Van der Burg demonstrated that temporal order judgment (TOJ) precision could be profoundly impaired by the mere presence of dynamic visual clutter elsewhere in the visual field. This study examines whether presenting target and distractor objects in different depth planes might ameliorate this remote temporal camouflage (RTC) effect. TOJ thresholds were measured under static and dynamic (flickering) distractor conditions. In Experiment 1, targets were presented at zero, crossed, or uncrossed disparity, with distractors fixed at zero disparity. Thresholds were significantly elevated under dynamic compared with static contextual conditions, replicating the RTC effect. Crossed but not uncrossed disparity targets improved performance in dynamic distractor contexts, which otherwise produce substantial RTC. In Experiment 2, the assignment of disparity was reversed: targets fixed at zero disparity; distractors crossed, uncrossed, or zero. Under these conditions, thresholds improved significantly in the nonzero distractor disparity conditions. These results indicate that presenting target and distractor objects in different planes can significantly improve TOJ performance in dynamic conditions. In Experiment 3, targets were each presented with a different sign of disparity (e.g., one crossed and the other uncrossed), with no resulting performance benefits. Results suggest that disparity can be used to alleviate the performance-diminishing effects of RTC, but only if both targets constitute a single and unique disparity-defined surface.

Projection-type integral imaging system using a three-dimensional screen composed of a lens array and a retroreflector film.

We propose an improved projection-type integral imaging system using a three-dimensional (3D) screen consisting of a lens array and a retroreflector film in this paper. The projection-type integral imaging system suffers from the disadvantage of low-visibility images because of the inherently small exit pupil size of the projector. In order to resolve this problem, we adopt a 3D screen to avoid the demerits of a diffuser screen, such as off-screen image blur and loss of parallax. To determine the appropriate configuration of the 3D screen in the system, a simulation based on a ray transfer matrix analysis method was performed. The results show that the 3D screen should be located near the central depth plane of the integral imaging system, which leads to the conclusion that only the real mode is available for the proposed system. Experiments to verify this configuration and the feasibility of the proposed system were conducted using a system constructed with a real mode integral imaging system including a convex mirror array, which can fundamentally eliminate the pseudoscopic problem.

Real-time sensing and three-dimensional display of far outdoor scenes based on asymmetric integral imaging

A new asymmetric integral imaging (AII) system for real-time pickup and three-dimensional (3-D) display of far outdoor scenes based on dynamic-pixel-mapping (DPM) is proposed. DPM is a digital process to transform the elemental images captured with a lens array into the perspective-variant object images (POIs) whose structures are matched with those of display lenses, where the orders of pixels in each POI are reversely mapped, and then capture a set of virtual elemental images (EIs) at the specific depth planes from the back-propagated POIs. This DPM enables an asymmetrical use of pickup and display lens arrays, allowing the long-ranged pickup of far outdoor scenes and their resolution-enhanced 3-D reconstruction. Experiments with a pair of pickup and display lens arrays whose pitches and focal lengths are given by 7.5mm, 30mm and 1.2mm, 8mm, respectively, show that the effective pickup-range and resolution of the proposed system have been increased up to 6m and 1600×1600 pixels, respectively, from 0.064m and 480×480 pixels of the conventional systems employing the same pickup and display lens arrays. In addition, experiments with an implemented test bed confirms that the proposed system can provide real-time 3-D images in 25 frames per second.

Wide Field Of View Varifocal Near-Eye Display Using See-Through Deformable Membrane Mirrors

Accommodative depth cues, a wide field of view, and ever-higher resolutions all present major hardware design challenges for near-eye displays. Optimizing a design to overcome one of these challenges typically leads to a trade-off in the others. We tackle this problem by introducing an all-in-one solution - a new wide field of view, gaze-tracked near-eye display for augmented reality applications. The key component of our solution is the use of a single see-through, varifocal deformable membrane mirror for each eye reflecting a display. They are controlled by airtight cavities and change the effective focal power to present a virtual image at a target depth plane which is determined by the gaze tracker. The benefits of using the membranes include wide field of view (100° diagonal) and fast depth switching (from 20 cm to infinity within 300 ms). Our subjective experiment verifies the prototype and demonstrates its potential benefits for near-eye see-through displays.

Fast and Forceful: Modulation of Response Activation Induced by Shifts of Perceived Depth in Virtual 3D Space.

Reaction time (RT) can strongly be influenced by a number of stimulus properties. For instance, there was converging evidence that perceived size rather than physical (i.e., retinal) size constitutes a major determinant of RT. However, this view has recently been challenged since within a virtual three-dimensional (3D) environment retinal size modulation failed to influence RT. In order to further investigate this issue in the present experiments response force (RF) was recorded as a supplemental measure of response activation in simple reaction tasks. In two separate experiments participants’ task was to react as fast as possible to the occurrence of a target located close to the observer or farther away while the offset between target locations was increased from Experiment 1 to Experiment 2. At the same time perceived target size (by varying the retinal size across depth planes) and target type (sphere vs. soccer ball) were modulated. Both experiments revealed faster and more forceful reactions when targets were presented closer to the observers. Perceived size and target type barely affected RT and RF in Experiment 1 but differentially affected both variables in Experiment 2. Thus, the present findings emphasize the usefulness of RF as a supplement to conventional RT measurement. On a behavioral level the results confirm that (at least) within virtual 3D space perceived object size neither strongly influences RT nor RF. Rather the relative position within egocentric (body-centered) space presumably indicates an object’s behavioral relevance and consequently constitutes an important modulator of visual processing.

The effects of the binocular disparity differences between targets and maskers on visual search.

A visual search for targets is facilitated when the target objects are on a different depth plane than other masking objects cluttering the scene. The ability of observers to determine whether one of four letters presented stereoscopically at four symmetrically located positions on the fixation plane differed from the other three was assessed when the target letters were masked by other randomly positioned and oriented letters appearing on the same depth plane as the target letters, or in front, or behind it. Three additional control maskers, derived from the letter maskers, were also presented on the same three depth planes: (1) random-phase maskers (same spectral amplitude composition as the letter masker but with the phase spectrum randomized); (2) random-pixel maskers (the locations of the letter maskers' pixel amplitudes were randomized); (3) letter-fragment maskers (the same letters as in the letter masker but broken up into fragments). Performance improved with target duration when the target-letter plane was in front of the letter-masker plane, but not when the target letters were on the same plane as the masker, or behind it. A comparison of the results for the four different kinds of maskers indicated that maskers consisting of recognizable objects (letters or letter fragments) interfere more with search and comparison judgments than do visual noise maskers having the same spatial frequency profile and contrast. In addition, performance was poorer for letter maskers than for letter-masker fragments, suggesting that the letter maskers interfered more with performance than the letter-fragment maskers because of the lexical activity they elicit.

A hierarchical methodology for urban facade parsing from TLS point clouds

The effective and automated parsing of building facades from terrestrial laser scanning (TLS) point clouds of urban environments is an important research topic in the GIS and remote sensing fields. It is also challenging because of the complexity and great variety of the available 3D building facade layouts as well as the noise and data missing of the input TLS point clouds. In this paper, we introduce a novel methodology for the accurate and computationally efficient parsing of urban building facades from TLS point clouds. The main novelty of the proposed methodology is that it is a systematic and hierarchical approach that considers, in an adaptive way, the semantic and underlying structures of the urban facades for segmentation and subsequent accurate modeling. Firstly, the available input point cloud is decomposed into depth planes based on a data-driven method; such layer decomposition enables similarity detection in each depth plane layer. Secondly, the labeling of the facade elements is performed using the SVM classifier in combination with our proposed BieS-ScSPM algorithm. The labeling outcome is then augmented with weak architectural knowledge. Thirdly, least-squares fitted normalized gray accumulative curves are applied to detect regular structures, and a binarization dilation extraction algorithm is used to partition facade elements. A dynamic line-by-line division is further applied to extract the boundaries of the elements. The 3D geometrical façade models are then reconstructed by optimizing facade elements across depth plane layers. We have evaluated the performance of the proposed method using several TLS facade datasets. Qualitative and quantitative performance comparisons with several other state-of-the-art methods dealing with the same facade parsing problem have demonstrated its superiority in performance and its effectiveness in improving segmentation accuracy.

3D compressive spectral integral imaging.

A novel compressive 3D imaging spectrometer based on the coded aperture snapshot spectral imager (CASSI) is proposed. By inserting a microlens array (MLA) into the CASSI system, one can capture spectral data of 3D objects in a single snapshot without requiring 3D scanning. The 3D spatio-spectral sensing phenomena is modelled by computational integral imaging in tandem with compressive coded aperture spectral imaging. A set of focal stack images is reconstructed from a single compressive measurement, and presented as images focused on different depth planes where the objects are located. The proposed optical system is demonstrated with simulations and experimental results.

Single-image full-focus reconstruction using depth-based deconvolution

In contrast with traditional extended depth-of-field approaches, we propose a depth-based deconvolution technique that realizes the depth-variant nature of the point spread function of an ordinary fixed-focus camera. The developed technique brings a single blurred image to focus at different depth planes which can be stitched together based on a depth map to output a full-focus image. Strategies to suppress the deconvolution’s ringing artifacts are implemented on three levels: block tiling to eliminate boundary artifacts, reference maps to reduce ringing initiated by sharp edges, and depth-based masking to mitigate artifacts raised by neighboring depth-transition surfaces. The performance is validated numerically for planar and multidepth objects.

A three-dimensional spatial characterization of the crossed-hands deficit

To perceive the location of touch in space, we integrate information about skin-location with information about the location of that body part in space. Most research investigating this process of tactile spatial remapping has used the so-called crossed-hands deficit, in which the ability to judge the temporal order of touches on the two hands is impaired when the arms are crossed. This posture induces a conflict between skin-based and tactile external spatial representations, specifically in the left-right dimension. Thus, it is unknown whether touch is affected by posture when spatial relations other than the right-left dimension are available. Here, we tested the extent to which the crossed-hands deficit is a measure of tactile remapping, reflecting tactile encoding in three-dimensional space. Participants judged the temporal order of tactile stimuli presented to crossed and uncrossed hands. The arms were placed at different elevations (up-down dimension; Experiments 1 and 2), or at different distances from the body in the depth plane (close-far dimension; Experiment 3). The crossed-hands deficit was reduced when other sources of spatial information, orthogonal to the left-right dimension (i.e., close-far, up-down), were available. Nonetheless, the deficit persisted in all conditions, even when processing of non-conflicting information in the close-far or up-down dimensions was enough to solve the task. Together, these results demonstrate that the processing underlying the crossed-hands deficit is related to the encoding of tactile localization in three-dimensional space, rather than related uniquely to the cost of processing information in the right-left dimension. Furthermore, the persistence of the crossing effect provides evidence for automatic integration of all available information during the encoding of tactile information.