Head-mounted Display Research Articles

Deformable patch-based garment design in immersive virtual reality

PurposeThis study aims to create and deform 3D garment apparel in an immersive virtual reality using head-mounted display and controllers. For this, adequate design methods for immersive virtual environment were explored and developed in order to confirm the suitability of the developed methods.Design/methodology/approachAn immersive virtual environment was prepared using Unreal Engine (UE) version 5.1 and Meta Human Lexi to create template garment that corresponds to the sizes of a human model. Dual quaternion skinning was adopted for pose deformation. For size deformation, patches were constructed with the measurement lines defined on Lexi. This patch-based approach was adopted not only for automatic generation but also for flat pattern projection of the template garment.FindingsThe research found that garment-making process can be brought into immersive virtual reality. Free use of one's hands and body made apparel deformation in an immersive environment conform with the real garment draping process.Originality/valueSimulating garment making in an immersive virtual reality has not previously been explored in detail. This research discovered, implemented and tested methods that best suit the environment where head-mounted display and controllers are essential in detail.

When virtual others are with me: exploring the influence of social presence in virtual reality wine tourism experiences

PurposeThe purpose of this study is to examine the role of social presence in enhancing positive behaviors in the virtual reality (VR)-based wine tourism context through an innovative approach.Design/methodology/approachTwo sequential experimental studies were conducted to test proposed hypotheses using Web and head-mounted display (HMD) VR formats. Specifically, Study 1 probed the influence of social presence on mental imagery, which subsequently impacted destination visit intention, drinking intent and memorable experience. Study 2 used a field experiment to explore the boundary effects of environmental cues (nature versus social) on social presence and a series of behavioral intentions using an HMD format.FindingsThe findings represent one of the first efforts to unravel the influence of social presence on positive behaviors through mental imagery and the moderating role of environmental cues.Research limitations/implicationsThis research enhances the understanding of wine tourism, drawing upon social presence theory and stimulus-organism-response framework.Practical implicationsThe outcomes provide valuable insights for wine tourism marketers in developing innovative marketing strategies by addressing the usage of social presence and environmental cues in a VR setting.Originality/valueTo the best of the authors’ knowledge, this study is the first to enrich the existing knowledge of wine tourism by exploring the role of social presence and environmental cues in both Web and HMD VR formats.

Virtual Reality to Reduce Pain, Fatigue, and Emotional Distress in People With Cancer: A Scoping Review.

Cancer and its treatments cause symptoms such as pain, fatigue, and emotional distress, which affect the quality of life of patients at different stages of cancer. Virtual reality (VR) has emerged as a promising alternative for addressing these symptoms by immersing patients in a virtual environment that isolates them from reality. To describe the existing evidence on the use of VR to improve pain, fatigue, and emotional distress in people with cancer at different stages of the cancer trajectory. A scoping review was conducted following the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews) guidelines. Search was conducted in PubMed and CINAHL until October 2023. Twenty-three primary studies related to the study topic were identified, and the following information was collected: type of VR (immersive vs nonimmersive) used in oncology services, purpose (distraction, training, or relaxation), devices used (head-mounted display, headset, or virtual controller), and the environment simulated (natural environment, skill games, or educational environment). The effects of VR on pain, fatigue, and emotional distress are described, highlighting its ability to mitigate these symptoms in the short term. Virtual reality has been shown to be an effective technique for reducing pain, fatigue, and emotional distress in people with cancer in the short term. Nurses can use VR as a complementary tool to promote virtual environments that improve the care for and therefore the quality of life of people with cancer.

Virtual Reality (VR) Technology in Laparoscopy Surgery: The Effect of Large Viewing Angle from the Head-Mounted Display (HMD) and Laparoscopic Performance

Background: A vital component of laparoscopic surgery is its display monitor. The viewing angle of the laparoscopic monitor impacts on the performance of the surgeon. Given that head-mounted display (HMD), an integral part of Virtual Reality (VR) technology, offers a large viewing angle, this project aimed to determine the potential benefits of using HMD in performing laparoscopic tasks. Methods: 54 laparoscopic/endoscopic clinicians performed a simulated laparoscopic task with three different monitors (HMD, 21-inch and 15-inch monitor) in a random order. Duration of task, distance travelled by graspers and manipulation errors were recorded and analyzed. Simulation Sickness Questionnaires (SSQs) were also given to participants. Results: There were statistically significant differences in all performance metrics recorded across all groups when compared to each other (P<0.05). HMD had the longest task completion duration followed by 15-inch and 21-inch monitors. However, the distance travelled by the graspers and error rate was lowest in the HMD group followed by 21-inch and 15-inch monitors. The SSQ questionnaire showed that eye strain and headache are the two most common transient side effects recorded by users of HMD. Conclusion: The study shows that HMD users perform fewer mistakes and are more precise with the movement of graspers compared to other groups. As such, HMD has the potential to enhance laparoscopic performance. HMD is cost effective and has the added benefit of improving surgical ergonomics. Further research should focus on ensuring that HMD is a viable and safe alternative to conventional laparoscopic monitor as well as in identifying barriers to adoption.

HazARdSnap: Gazed-Based Augmentation Delivery for Safe Information Access While Cycling.

During cycling activities, cyclists often monitor a variety of information such as heart rate, distance, and navigation using a bike-mounted phone or cyclocomputer. In many cases, cyclists also ride on sidewalks or paths that contain pedestrians and other obstructions such as potholes, so monitoring information on a bike-mounted interface can slow the cyclist down or cause accidents and injury. In this article, we present HazARdSnap, an augmented reality-based information delivery approach that improves the ease of access to cycling information and at the same time preserves the user's awareness of hazards. To do so, we implemented real-time outdoor hazard detection using a combination of computer vision and motion and position data from a head mounted display (HMD). We then developed an algorithm that snaps information to detected hazards when they are also viewed so that users can simultaneously view both rendered virtual cycling information and the real-world cues such as depth, position, time to hazard, and speed that are needed to assess and avoid hazards. Results from a study with 24 participants that made use of real-world cycling and virtual hazards showed that both HazARdSnap and forward-fixed augmented reality (AR) user interfaces (UIs) can effectively help cyclists access virtual information without having to look down, which resulted in fewer collisions (51% and 43% reduction compared to baseline, respectively) with virtual hazards.

Examining the effect of locomotion techniques on virtual prototype assessment: Gaze analysis using a Head-Mounted Display

Improvements in the performance and graphical quality of Head-Mounted Displays (HMDs) have led to their increasing use in Virtual Reality (VR) for product presentation and virtual prototype (VP) evaluations. Various locomotion techniques in VR make it possible to move through a virtual scenario and approach the VP for evaluation purposes. The integration of eye-tracking devices into recent HMDs allows the trajectory and gaze behavior of observers to be reported during the evaluation, often more objectively than self-report questionnaires. However, very few studies have used physiological measures for the evaluation of products embedded in VR environments. Therefore, this paper offers a study in which 95 people evaluated three VPs of street furniture presented in their environment of use using Meta Quest Pro headset and explored through teleport and natural walking. The influence of the locomotion techniques on the ratings recorded using a semantic differential, sense of presence, cybersickness, and the role of eye-tracking in understanding gaze behavior while evaluating products' Areas of Interest (AOIs), are investigated. This study found no evidence that the way of approaching the product influences the evaluation of some of its features, overall product evaluation, confidence in responses, sense of presence, or cybersickness differently. On the other hand, this work evidences that the locomotion technique had an impact on how the user approached the products, which could significantly influence the viewing time of some AOIs. The study revealed that the most observed AOIs coincided with those parts closely related to important features, generally located at the top of the products, so paying special attention to these parts when designing and evaluating similar VPs is recommended.

HeadShift: Head Pointing with Dynamic Control-Display Gain

Head pointing is widely used for hands-free input in head-mounted displays (HMDs). The primary role of head movement in an HMD is to control the viewport based on absolute mapping of head rotation to the 3D environment. Head pointing is conventionally supported by the same 1:1 mapping of input with a cursor fixed in the centre of the view, but this requires exaggerated head movement and limits input granularity. In this work, we propose to adopt dynamic gain to improve ergonomics and precision, and introduce the HeadShift technique. The design of HeadShift is grounded in natural eye-head coordination to manage control of the viewport and the cursor at different speeds. We evaluated HeadShift in a Fitts’ Law experiment and on three different applications in VR, finding the technique to reduce error rate and effort. The findings are significant as they show that gain can be adopted effectively for head pointing while ensuring that the cursor is maintained within a comfortable eye-in-head viewing range.

Virtual reality vs. Tablet video for venipuncture education in children: A randomized clinical trial.

Pediatric patients usually experience high levels of pain and distress due to venipuncture. This randomised study aimed to evaluate the effects of virtual reality-based preprocedural education in comparison with video-based education in terms of pain and distress experienced by children scheduled to undergo venipuncture. Ninety children aged 4-8 years who were scheduled to undergo venipuncture surgery were randomly assigned to either a video or virtual reality group. Children in the video group received preprocedural education on venipuncture via a video displayed on a tablet and those in the virtual reality group received the same education via a head-mounted virtual reality display unit. The educational content for the two groups was identical. An independent assessor blinded to the group assignment observed the children's behavior and determined their Children's Hospital of Eastern Ontario Pain Scale scores, parental satisfaction score, procedure-related outcomes, venipuncture time, number of repeated procedures and difficulty score for the procedure. The virtual reality group experienced less pain and distress, as indicated by their Children's Hospital of Eastern Ontario Pain Scale scores compared with the video group (5.0 [5.0-8.0] vs. 7.0 [5.0-9.0], P = 0.027). There were no significant intergroup differences in parental satisfaction scores or procedure-related outcomes. For pediatric patients scheduled to undergo venipuncture, preprocedural education via a head-mounted display for immersive virtual reality was more effective compared with video-based education via a tablet in terms of reducing pain and distress.

Hello, is it me you're Stopping for? The Effect of external Human Machine Interface Familiarity on Pedestrians' Crossing Behaviour in an Ambiguous Situation.

We investigated how different deceleration intentions (i.e. an automated vehicle either decelerated for leading traffic or yielded for pedestrians) and a novel (Slow Pulsing Light Band - SPLB) or familiar (Flashing Headlights - FH) external Human Machine Interface (eHMI) informed pedestrians' crossing behaviour. The introduction of SAE Level 4 Automated Vehicles (AVs) has recently fuelled interest in new forms of explicit communication via eHMIs, to improve the interaction between AVs and surrounding road users. Before implementing these eHMIs, it is necessary to understand how pedestrians use them to inform their crossing decisions. Thirty participants took part in the study using a Head-Mounted Display. The independent variables were deceleration intentions and eHMI design. The percentage of crossings, collision frequency and crossing initiation time across trials were measured. Pedestrians were able to identify the intentions of a decelerating vehicle, using implicit cues, with more crossings made when the approaching vehicles were yielding to them. They were also more likely to cross when a familiar eHMI was presented, compared to a novel one or no eHMI, regardless of the vehicle's intention. Finally, participants learned to take a more cautious approach as trials progressed, and not to base their decisions solely on the eHMI. A familiar eHMI led to early crossings regardless of the vehicle's intention but also led to a higher collision frequency than a novel eHMI. To achieve safe and acceptable interactions with AVs, it is important to provide eHMIs that are congruent with road users' expectations.

Muscular tension in ear surgeons during cochlear implantations: does a new microscope improve musculoskeletal complaints?

Musculoskeletal complaints and fatigue are commonly described symptoms in daily work of ear nose and throat surgeons using a microscope. Long ear surgical procedures are associated with prolonged microscope use, which can lead to unconsciously tense and uncomfortable body posture. The digital microscope RoboticScope® allows visualization of surgical site through a Head-Mounted Display, independent from camera head and is therefore much easier on the back, as the sitting position can be adjusted flexibly. Aim of the prospective study was to investigate to what extent the use of a RoboticScope® changes the tension of neck and shoulder muscles of the surgeons in comparison to a conventional tripod microscope. For this purpose, the electric activity of neck and shoulder muscles of surgeons was recorded using surface electromyography during cochlear implantations. Electrical potentials were derived via electrodes on neck and shoulder muscles. The basic tension of those muscles was measured in relaxed position before and after surgery. During microscope use the tension was continued to be measured. A questionnaire recorded parameters such as level of difficulty of operation as well as patient data. Results from 58 operations, 33 of which were performed using a conventional microscope and 25 using a Head-Mounted Display, show a significant reduction in muscular tension during surgery by 40% (p < 0.001) in experienced ear surgeons when using a RoboticScope®, regardless of the surgeon. Our results are in line with the relevance of preventive measures to avoid acute and chronic work-related illnesses/symptoms described in literature.

PilotAR: Streamlining Pilot Studies with OHMDs from Concept to Insight

Pilot studies in HCI research serve as a cost-effective approach to validate potential ideas and identify impactful findings before extensive studies. Yet, the additional requirements of AR/MR, such as multi-view observations and increased multitasking, make it challenging to conduct pilot studies effectively, hindering innovations in this field. Based on interviews with 12 AR/MR researchers, we identified the key challenges associated with conducting AR/MR pilot studies with Optical See-Through Head-Mounted Displays (OST-HMDs, OHMDs), including the inability to observe and record in-context user interactions, increased task load, and difficulties with in-context data analysis and discussion. To tackle these challenges, we introduce PilotAR, a desktop-based tool designed iteratively to enhance OHMD-based AR/MR pilot studies. PilotAR facilitates data collection via live first-person and third-person views, multi-modal annotations, and flexible wizarding interfaces. It also accommodates multi-experimenter settings, streamlines the study process with configurable workflows and shortcuts, records annotated data, and eases results sharing. Formative testing, conducted using three case studies, has highlighted the significant benefits of PilotAR, as well as its potential for further development and refinement.

Decline in Sensory Integration in Old Age and Its Related Functional Brain Connectivity Correlates Observed during a Virtual Reality Task.

Sensory integration is an essential human function whose decline impacts quality of life, particularly in older adults. Herein, we propose an arm-reaching task based on a virtual reality head-mounted display system to assess sensory integration in daily life, and we examined whether reaching task performance was associated with resting-state functional connectivity (rsFC) between the brain regions involved in sensory integration. We hypothesized that declining sensory integration would affect performance during a reaching task with multiple cognitive loads. Using a task in which a young/middle-aged group showed only small individual differences, older adults showed large individual differences in the gap angle between the reaching hand and the target position, which was used to assess sensory integration function. Additionally, rsfMRI data were used to identify correlations between rsFC and performance in older adults, showing that performance was correlated with connectivity between the primary motor area and the left inferior temporal gyrus and temporo-occipital region. Connectivity between areas is related to visuomotor integration; thus, the results suggest the involvement of visuomotor integration in the decline of sensory integration function and the validity of the gap angle during this VR reaching task as an index of functional decline.

Social virtual reality: systematic review of virtual teamwork with head-mounted displays

PurposeSocial virtual reality (SVR) has emerged as a solution for the facilitation of remote and distributed teamwork, promising to overcome challenges faced in virtual 2D communication channels. The purpose of this study is to systematically review empirical articles dealing with the application of SVR with head-mounted displays (HMDs) in teams.Design/methodology/approachThis study reviewed 40 empirical papers on teamwork that address the application of SVR using HMD in teams. The authors present results from the extracted data and identify themes that indicate the current state of research in this field.FindingsBased on data obtained from the journal articles, the authors identified three themes underlying current research. They are features and applications of SVR technology, collaboration dynamics and team performance in SVR and educational and professional training and learning. The results show that SVR enhances team collaboration and collaborative learning.Practical implicationsSVR can aid in improving teamwork and team outcomes. In its current stage, SVR is complementary to 2D technology and not likely to replace traditional tools. Findings suggest that the technology is promising for collaboration and learning in organisations, especially in collaborative design tasks and simulations.Originality/valueTo the best of the authors’ knowledge, this paper is one of the first studies to look at research emerging from the nascent area of SVR and summarise the current state of research. This study provides managerial implications and identifies future research areas for SVR in teamwork and learning.

From microscope to head-mounted display: integrating hand tracking into microsurgical augmented reality.

The operating microscope plays a central role in middle and inner ear procedures that involve working within tightly confined spaces under limited exposure. Augmented reality (AR) may improve surgical guidance by combining preoperative computed tomography (CT) imaging that can provide precise anatomical information, with intraoperative microscope video feed. With current technology, the operator must manually interact with the AR interface using a computer. The latter poses a disruption in the surgical flow and is suboptimal for maintaining the sterility of the operating environment. The purpose of this study was to implement and evaluate free-hand interaction concepts leveraging hand tracking and gesture recognition as an attempt to reduce the disruption during surgery and improve human-computer interaction. An electromagnetically tracked surgical microscope was calibrated using a custom 3D printed calibration board. This allowed the augmentation of the microscope feed with segmented preoperative CT-derived virtual models. Ultraleap's Leap Motion Controller 2 was coupled to the microscope and used to implement hand-tracking capabilities. End-user feedback was gathered from a surgeon during development. Finally, users were asked to complete tasks that involved interacting with the virtual models, aligning them to physical targets, and adjusting the AR visualization. Following observations and user feedback, we upgraded the functionalities of the hand interaction system. User feedback showed the users' preference for the new interaction concepts that provided minimal disruption of the surgical workflow and more intuitive interaction with the virtual content. We integrated hand interaction concepts, typically used with head-mounted displays (HMDs), into a surgical stereo microscope system intended for AR in otologic microsurgery. The concepts presented in this study demonstrated a more favorable approach to human-computer interaction in a surgical context. They hold potential for a more efficient execution of surgical tasks under microscopic AR guidance.

Remote proctoring during structural heart procedures: Towarda widespread diffusion of knowledge using mixed reality.

Despite its wide diffusion in surgical procedures, very few experiences are reported so far about the possible applications of remote proctoring (RP) in structural heart interventions. Our aim was to demonstrate the feasibility, safety, and efficacy of RP during transcatheter edge-to-edge repair using a mixed reality-based head-mounted display (HMD). Two users (a doctor and a proctor), wearing HMDs, were connected through a 5G network, allowing them to share audio and video signals. During the procedure, the proctor was located in a contiguous room, without any direct connection with the operator. The primary endpoint was noninferiority of HMD-mediated interaction if compared with direct in-person interaction. It was assessed using an operator-proctor interaction test based on six questions asked by the doctor to the proctor on six key procedural steps. Between September 2021 and April 2022, ninepatients were enrolled in the study (15 clips delivered). The primary endpoint was reached in 9/15 clips implanted (60%). The first threefailures were due to a chat software problem, and the others were linked with a 5G network malfunction. However, the HMD and its mixed reality tools were well tolerated by the operators, and no interference with the procedure was registered in any of the cases. In our experience, RP during structural heart interventions is feasible and efficacious. Wearing the HMD is comfortable and does not affect safety and effectiveness of interventional procedures. However, especially if consideringa wide geographic use of this tool, a stable internet connection is imperative.

Evaluation of Remote Surgical Hands-on Training in Veterinary Education Using a Hololens Mixed Reality Head-Mounted Display

Evaluation of Remote Surgical Hands-on Training in Veterinary Education Using a Hololens Mixed Reality Head-Mounted Display

Democratizing EEG: Embedding Electroencephalography in a Head-Mounted Display for Ubiquitous Brain-Computer Interfacing

Open hardware and the need for ecologically valid measurements drive the Electroencephalography (EEG) democratization movement—EEG has been steadily transcending the boundaries of clinical research, making its way into interdisciplinary fields. In Human-Computer Interaction (HCI), EEG is used to measure cognitive workload and infer cognitive processes for building cognition-aware systems. We describe and evaluate our BCIglass prototype where EEG electrodes are embedded in the frame of a mainstream Head-Mounted Display (HMD) to create a skull-peripheral topology. We devised a lab study with 34 participants who completed seven established cognitive tasks. Then, we conducted a pilot field study with one participant to test BCIglass in everyday-life settings. Our findings demonstrate that BCIglass captures EEG activity in a manner comparable to a research-grade EEG-cap system. Our topology infers the cognitive task at hand, and the underlying cognitive process(es) by proxy, with an accuracy of ∼80% and only three electrodes at the skull periphery. Embedding EEG electrodes in lightweight HMDs represents a promising approach in the quest to achieve ubiquitous brain-computer interfacing in real-world settings.

Perception-Driven Soft-Edge Occlusion for Optical See-Through Head-Mounted Displays.

Systems with occlusion capabilities, such as those used in vision augmentation, image processing, and optical see-through head-mounted display (OST-HMD), have gained popularity. Achieving precise (hard-edge) occlusion in these systems is challenging, often requiring complex optical designs and bulky volumes. On the other hand, utilizing a single transparent liquid crystal display (LCD) is a simple approach to create occlusion masks. However, the generated mask will appear defocused (soft-edge) resulting in insufficient blocking or occlusion leakage. In our work, we delve into the perception of soft-edge occlusion by the human visual system and present a preference-based optimal expansion method that minimizes perceived occlusion leakage. In a user study involving 20 participants, we made a noteworthy observation that the human eye perceives a sharper edge blur of the occlusion mask when individuals see through it and gaze at a far distance, in contrast to the camera system's observation. Moreover, our study revealed significant individual differences in the perception of soft-edge masks in human vision when focusing. These differences may lead to varying degrees of demand for mask size among individuals. Our evaluation demonstrates that our method successfully accounts for individual differences and achieves optimal masking effects at arbitrary distances and pupil sizes.

The acute hypoalgesic effects of active head-mounted display virtual reality games.

The purpose of this study was to determine: (1) whether physically active virtual reality (VR) games exert an acute hypoaglesic effect on the thigh and bicep compared to a non-active VR game and an exercise only condition matched for exercise intensity in healthy individuals, and (2) whether movement variables during gameplay are associated with the hypoalgesic effect of the games. Twenty young adults completed five separate study sessions, with each session devoted to playing one head-mounted display VR game or stationary cycling for 15 minutes. The games included Holopoint at level 2 and level 3, Hot Squat, and Relax Walk. Pressure pain thresholds at the thigh and bicep were measured pre and post VR gameplay and cycling. Participants wore a heart rate monitor and accelerometers on the wrist and thigh during play to measure the intensity and quantity of movement. Repeated measures ANOVAs revealed that pressure pain thresholds on the bicep increased from pre to posttest for each condition. The results also revealed that pressure pain thresholds on the thigh increased only for the conditions eliciting the greatest cardiovascular response, which included Holopoint at level 3, Hot Squat, and cycling. Bivariate correlations indicated that moderate to vigorous physical activity of the thigh was associated with pain reduction at the thigh during Holopoint. These results revealed that active VR games and exercise exerted a more widespread hypoalgesic effect compared to the non-active VR game, which was likely driven in part by the intensity and quantity of movement during gameplay.

Effects of invisible body and optic flow on experience of users voluntarily walking in a VR environment

Studies have demonstrated that a multi-modal virtual reality (VR) system can enhance the realism of virtual walking. However, a few studies explore the body awareness altered by visual presentation of virtual body and optic flow during locomotion in VR. This study investigated the impact of invisible body and optic flow on experience of users voluntarily walking in a camera-image VR environment. Participants wearing a head-mounted display performed six-step walking at their own timing. Three experimental conditions providing visible body and optic flow as a baseline, invisible body and optic flow, and invisible body and no flow, were conducted on three different days. We found that losing visual body per se decreased the feeling of being-there-now. However, providing continuous optic flow maintained virtual presence equivalent to the baseline in terms of immersion and natural walking, as opposed to providing discontinuous flow. We discussed these results in association with body awareness.