Stereoscopic Head-mounted Display Research Articles

Quasi-3D: reducing convergence effort improves visual comfort of head-mounted stereoscopic displays

The diffusion of virtual reality urges to solve the problem of vergence-accommodation conflict arising when viewing stereoscopic displays, which causes visual stress. We addressed this issue with an approach based on reducing ocular convergence effort. In virtual environments, vergence can be controlled by manipulating the binocular separation of the virtual cameras. Using this technique, we implemented two quasi-3D conditions characterized by binocular image separations intermediate between 3D (stereoscopic) and 2D (monoscopic). In a first experiment, focused on perceptual aspects, ten participants performed a visuo-manual pursuit task while wearing a head-mounted display (HMD) in head-constrained (non-immersive) condition for an overall exposure time of ~ 7 min. Passing from 3D to quasi-3D and 2D conditions, progressively resulted in a decrease of vergence eye movements—both mean convergence angle (static vergence) and vergence excursion (dynamic vergence)—and an increase of hand pursuit spatial error, with the target perceived further from the observer and larger. Decreased static and dynamic vergence predicted decreases in asthenopia trial-wise. In a second experiment, focused on tolerance aspects, fourteen participants performed a detection task in near-vision while wearing an HMD in head-free (immersive) condition for an overall exposure time of ~ 20 min. Passing from 3D to quasi-3D and 2D conditions, there was a general decrease of both subjective and objective visual stress indicators (ocular convergence discomfort ratings, cyber-sickness symptoms and skin conductance level). Decreased static and dynamic vergence predicted the decrease in these indicators. Remarkably, skin conductance level predicted all subjective symptoms, both trial-wise and session-wise, suggesting that it could become an objective replacement of visual stress self-reports. We conclude that relieving convergence effort by reducing binocular image separation in virtual environments can be a simple and effective way to decrease visual stress caused by stereoscopic HMDs. The negative side-effect—worsening of spatial vision—arguably would become unnoticed or compensated over time. This initial proof-of-concept study should be extended by future large-scale studies testing additional environments, tasks, displays, users, and exposure times.

Two is Better Than One. Improved Attention Guiding in AR by Combining Techniques.

Search tasks in production environments have been shown to be significantly improved by attention guiding techniques in augmented reality (AR). In mobile use cases, head-mounted displays support context-sensitive and flexible search tasks. To improve search time and navigation efficiency, first concepts like attention funnel and spherical wave based guidance techniques have been tested successfully in comparison to simple arrow navigation. Existing research indicates that for two-dimensional interfaces, a differentiation into coarse and fine navigation will result in further performance improvements. In this article, a laboratory study is presented with a stereoscopic head-mounted display that examines whether an additional arrow guiding technique for coarse navigation could improve both search time and efficiency of existing techniques. Results show objectively superior performance for one of the combined techniques in comparison to the unimodal attention guiding techniques and subjective preference by the participants. Further research should investigate the performance in a real environment.

Virtual reality framework for editing and exploring medial axis representations of nanometric scale neural structures

We present a novel virtual reality (VR) based framework for the exploratory analysis of nanoscale 3D reconstructions of cellular structures acquired from rodent brain samples through serial electron microscopy. The system is specifically targeted on medial axis representations (skeletons) of branched and tubular structures of cellular shapes, and it is designed for providing to domain scientists: i) effective and fast semi-automatic interfaces for tracing skeletons directly on surface-based representations of cells and structures, ii) fast tools for proofreading, i.e., correcting and editing of semi-automatically constructed skeleton representations, and iii) natural methods for interactive exploration, i.e., measuring, comparing, and analyzing geometric features related to cellular structures based on medial axis representations. Neuroscientists currently use the system for performing morphology studies on sparse reconstructions of glial cells and neurons extracted from a sample of the somatosensory cortex of a juvenile rat. The framework runs in a standard PC and has been tested on two different display and interaction setups: PC-tethered stereoscopic head-mounted display (HMD) with 3D controllers and tracking sensors, and a large display wall with a standard gamepad controller. We report on a user study that we carried out for analyzing user performance on different tasks using these two setups.

Audio in VR: Effects of a Soundscape and Movement-Triggered Step Sounds on Presence.

For effective virtual realities, “presence,” the feeling of “being there” in a virtual environment (VR), is deemed an essential prerequisite. Several studies have assessed the effect of the (non-)availability of auditory stimulation on presence, but due to differences in study design (e.g., virtual realities used, types of sounds included, rendering technologies employed), generalizing the results and estimating the effect of the auditory component is difficult. In two experiments, the influence of an ambient nature soundscape and movement-triggered step sounds were investigated regarding their effects on presence. In each experiment, approximately forty participants walked on a treadmill, thereby strolling through a virtual park environment reproduced via a stereoscopic head-mounted display (HMD), while the acoustical environment was delivered via noise-canceling headphones. In Experiment 1, conditions with the ambient soundscape and the step sounds either present or absent were combined in a 2 × 2 within-subjects design, supplemented with an additional “no-headphones” control condition. For the synchronous playback of step sounds, the probability of a step being taken was estimated by an algorithm using the HMD's sensor data. The results of Experiment 1 show that questionnaire-based measures of presence and realism were influenced by the soundscape but not by the reproduction of steps, which might be confounded with the fact that the perceived synchronicity of the sensor-triggered step sounds was rated rather low. Therefore, in Experiment 2, the step-reproduction algorithm was improved and judged to be more synchronous by participants. Consequently, large and statistically significant effects of both kinds of audio manipulations on perceived presence and realism were observed, with the effect of the soundscape being larger than that of including footstep sounds, possibly due to the remaining imperfections in the reproduction of steps. Including an appropriate soundscape or self-triggered footsteps had differential effects on subscales of presence, in that both affected overall presence and realism, while involvement was improved and distraction reduced by the ambient soundscape only.

Development and evaluation of a head-mounted display system based on stereoscopic images and depth algorithms for patients with visual impairment

In this work, we developed a wearable system using a commercial stereoscopic head-mounted display. We compared depth first-then contour (df-tc) and contour first-then depth (cf-td) algorithms in terms of images processed per second rate versus the window size and sweep values. Likewise, we performed a comparison of several edge detection methods in the same terms. The developed technical aid was clinically tested. We evaluated the preferred walking speed and the walking speed with and without the devices during three test circuits. For comparative analysis of anxiety levels, we recorded patients’ heart rate data before, during and after the test. The system has proven its potential for enhancing the patients’ mobility and reducing the level of anxiety related to movement activities.

Egocentric biases in comparative volume judgments of rooms.

The elongation of a figure or object can induce a perceptual bias regarding its area or volume estimation. This bias is notable in Piagetian experiments in which participants tend to consider elongated cylinders to contain more liquid than shorter cylinders of equal volume. We investigated whether similar perceptual biases could be found in volume judgments of surrounding indoor spaces and whether those judgments were viewpoint dependent. Participants compared a variety of computer-generated rectangular rooms with a square room in a psychophysical task. We found that the elongation bias in figures or objects was also present in volume comparison judgments of indoor spaces. Further, the direction of the bias (larger or smaller) depended on the observer's viewpoint. Similar results were obtained from a monoscopic computer display (Experiment 1) and stereoscopic head-mounted display with head tracking (Experiment 2). We used generalized linear mixed-effect models to model participants' volume judgments using a function of room depth and width. A good fit to the data was found when applying weight on the depth relative to the width, suggesting that participants' judgments were biased by egocentric properties of the space. We discuss how biases in comparative volume judgments of rooms might reflect the use of simplified strategies, such as anchoring on one salient dimension of the space.

A Virtual Reality System for Training Operators

This paper presents an immersive system for virtual reality-based training of operators. Recent studies show that virtual reality simulations may be helpful in different fields, namely, industry and education. In this particular case we present a simulator for the training of tower cranes operators reducing their costs and risks. This immersive system integrates real-time 3D computer graphics with other modalities, such as, user pose tracking, stereoscopic head-mounted display, pendant remote controller, physics behaviour, and spatial sound, in order to create a more realistic and immersive experience to the user. The results demonstrate two different simulator setups, one fixed and one portable, where the fixed offer a larger traceable area and the portable can be moved wherever is needed. The simulator is robust, reliable and provides an experience very similar to reality. Furthermore, it is possible to extend the simulation to other types of cranes and different types of scenarios.

The Convergence Effect: Real and Virtual Encounters in Augmented Reality Art

The Convergence Effect: Real and Virtual Encounters in Augmented Reality Art

An immersive simulation system for provoking and analyzing cataplexy.

Cataplexy, a sudden loss of voluntary muscle control, is one of the hallmark symptoms of narcolepsy, a sleep disorder characterized by excessive daytime sleepiness. Cataplexy is usually triggered by strong, spontaneous emotions, such as laughter, surprise, fear or anger, and is more common in times of stress. The Sleep Disorders Unit and the Biomedical Imaging Resource at Mayo Clinic are developing interactive display technology for reliably inducing cataplexy during clinical monitoring. The use of immersive displays may help bypass patient defenses, and game-like "unreality" allows introduction of surprising, threatening, or humorous elements, with little risk of offending patients. The project is referred to as the "Cataplexy/Narcolepsy Activation Program", or CatNAP. We have developed an automobile driving simulation to allow the introduction of humorous, surprising, or stress-inducing events and objects as the patient attempts to navigate a simulated vehicle through a virtual town. The patient wears a stereoscopic head-mounted display, by which he views the virtual town through the windows of his simulated vehicle. The vehicle is controlled via a driving simulator steering wheel and pedal cluster. The patient is instructed to drive his vehicle to another location in town, given initial directions and street signs. As he attempts to accomplish the task, various objects, sounds or conditions occur which may distract, startle, frustrate or cause laughter; responses which may trigger a cataplectic episode. The patient can be monitored by reflex tests and EMG recordings during the driving experience. An evaluation phase with volunteer patients previously diagnosed with cataplexy has been completed. The goal of these trials was to gain insight from the volunteers as to improvements that could be made to the simulation. All patients that participated in the evaluation phase have been under a physician's care for a number of years and control their cataplexy with medication. We believe this is a novel and innovative approach to a difficult problem. CatNAP is a compelling example of the potentially effective application of virtual reality technology to an important clinical problem that has resisted previous approaches. Preliminary results suggest that an immersive simulation system like CatNAP will be able to reliably induce cataplexy in a controlled environment. The project is continuing through a final stage of refinement prior to conducting a full clinical study.

Collaborative Aerial-Drawing System for Supporting Co-Creative Communication

This paper describes the collaborative augmented reality (AR) system with which multiple users can handwrite 3D lines in the air simultaneously and manipulate the lines directly in the real world. In addition, we propose a new technique for co-creative communication utilizing the 3D drawing activity. Up to now, the various 3D user interfaces have been proposed. Although most of them aim to solve the specific problems in the virtual environments, the possibility of the 3D drawing expression has not been explored yet. Accordingly, we paid special attention to the interaction with the real objects in daily life, and considered to manipulate real objects and 3D lines without any distinctions by the same action. The developed AR system consists of a stereoscopic head-mounted display, a drawing tool, 6DOF sensors measuring three-dimensional position and Euler angles, and the 3D user interface, which enables to push, grasp and pitch 3D lines directly by use of the drawing tool. Additionally users can pick up desired color from either a landscape or a virtual line through the direct interaction with this tool. For sharing 3D lines among multiple users at the same place, the distributed-type AR system has been developed that mutually sends and receives drawn data between systems. With the developed system, users can proceed to design jointly in the real space through arranging each 3D drawing by direct manipulation. Moreover, a new application to the entertainment has become possible to play sports like catch, fencing match, or the like.

Reality Augmentation for Medical Procedures: System Architecture, Single Camera Marker Tracking, and System Evaluation

Augmented Reality is an emerging technology that seeks to enhance a user's view by overlaying graphical information. We developed a prototype AR system geared for medical applications. It is built around a stereoscopic head-mounted display of the video-see-through variety. The newest generation of this prototype system exhibits high performance on a standard PC platform. Stereoscopic video images are augmented with medical graphics in real-time at 30 frames per second and with XGA (1024× 768) resolution. The system provides a compelling AR perception: the graphics appears firmly anchored in the scene--there is no time lag between video and graphics or any apparent jitter of the graphics. With the head-mounted display, the user has a natural and direct access to understanding the 3D structure of the scene, based on both stereo and kinetic depth cues. In the present paper, we describe in detail the architecture and several features of the AR prototype system. Head tracking is accomplished with a single-camera system, with the dedicated tracker camera placed on the head-mounted display. This configuration is the foundation of achieving a high-accuracy graphics overlay. We are now exploring the use of the prototype system for a variety of medical applications. This paper gives an overview over the pre-clinical tests that we have performed for interventional guidance. Overall, the feedback has been very positive and encouraging, and we are continuing to work towards realizing the clinical potential of the technology.

Stereoscopic head-mounted display incorporated into microsurgical procedures: technical note.

We have previously evaluated video technologies that have allowed for the use of electronic imaging during microneurosurgical and endoscopic procedures. This stereoscopic camera set and monitor/recording system allows for the use of stereoscopic images during surgery by the primary surgical assistant and ancillary personnel. It also allows for stereoscopic recording and playback using a wide video home system format. We describe a novel prototype of a head-mounted display (HMD) that allows the surgical team to simultaneously visualize the surgical field stereoscopically and includes picture-in-picture, voice control, and stereoscopic recording capabilities. A stereoscopic HMD with 640 x 480 (video graphics array) pixel resolution and the ability to display 24-bit images has been designed. This device weighs 900 g. It is interfaceable with common video display formats. This stereoscopic HMD is being evaluated in a prospective multicenter trial of open microsurgical and endoscopic minimally invasive procedures. In our experience to date, there have been no equipment failures or complications attributable to the use of the display system. The equipment was well accepted by users who reported significant benefits in visualization. The use of a stereoscopic HMD may result in improved efficiency and safety in both endoscopic and open microsurgical procedures. We have verified that the HMD is comfortable during the course of a surgical procedure, is reliable, and allows for accessibility to the operative field with an excellent field of view and three-dimensional perception. Positioning and dexterity within the operative field are also enhanced. Additional uses relate to surgical training, multimodal information display, and operative rehearsals.

Stereoscopic Head-mounted Display Incorporated into Microsurgical Procedures: Technical Note

Stereoscopic Head-mounted Display Incorporated into Microsurgical Procedures: Technical Note

Stereoscopic Head-mounted Display Incorporated into Microsurgical Procedures: Technical Note

Stereoscopic Head-mounted Display Incorporated into Microsurgical Procedures: Technical Note

Stereoscopic Head-mounted Display Incorporated into Microsurgical Procedures: Technical Note

Stereoscopic Head-mounted Display Incorporated into Microsurgical Procedures: Technical Note

Stereoscopic Head-mounted Display Incorporated into Microsurgical Procedures: Technical Note

Stereoscopic Head-mounted Display Incorporated into Microsurgical Procedures: Technical Note

Head-mounted display system for microneurosurgery.

Advances in video technology have enabled the incorporation of high-resolution electronic imaging of an operative field during microneurosurgical and endoscopic procedures. While current video-based monoscopic camera and monitor systems produce an excellent image, the third dimension is lacking and the operative field and image are at different locations. We describe a head-mounted display (HMD) that allows the surgical team to simultaneously visualize a surgical field and a high-resolution magnified stereoscopic image which includes a picture-in-picture capability. Stereoscopic HMDs should result in improved efficiency and safety in both endoscopic and open microsurgical procedures. Additional uses relate to surgical training and operative rehearsals.

A Human Factors Simulation Tool for Stereoscopic Displays

The increasing complexity of advanced display systems places increased importance on simulation tools. Display simulation tools allow the development of displays with the desired balance of image quality and end cost while reducing the time and cost otherwise associated with iterative testing of physical prototypes. In addition, simulation offers the advantage of permitting demonstration of display systems to targeted users and decision makers early in the display design cycle, allowing for more feedback in the design process. A new display simulation tool is described in this paper with the capability of simulating advanced electronic stereoscopic displays. To illustrate the use of this tool, an evaluation of a simulated stereoscopic head-mounted display is reported. Surgeons viewed stereoscopic laparoscopic imagery and rated the acceptability of display gray scale, resolution, and field of view. Clear preferences were seen for levels of each of these parameters. Simulations such as the one described in this report play an invaluable role in defining acceptable design parameters prior to commitment to display production.