HTC Vive Pro Research Articles

AffectiVR: A Database for Periocular Identification and Valence and Arousal Evaluation in Virtual Reality

This study introduces AffectiVR, a dataset designed for periocular biometric authentication and emotion evaluation in virtual reality (VR) environments. To maximize immersion in VR environments, interactions must be seamless and natural, with unobtrusive authentication and emotion recognition technologies playing a crucial role. This study proposes a method for user authentication by utilizing periocular images captured by a camera attached to a VR headset. Existing datasets have lacked periocular images acquired in VR environments, limiting their practical application. To address this, periocular images were collected from 100 participants using the HTC Vive Pro and Pupil Labs infrared cameras in a VR environment. Participants also watched seven emotion-inducing videos, and emotional evaluations for each video were conducted. The final dataset comprises 1988 monocular videos and corresponding self-assessment manikin (SAM) evaluations for each experimental video. This study also presents a baseline study to evaluate the performance of biometric authentication using the collected dataset. A deep learning model was used to analyze the performance of biometric authentication based on periocular data collected in a VR environment, confirming the potential for implicit and continuous authentication. The high-resolution periocular images collected in this study provide valuable data not only for user authentication but also for emotion evaluation research. The dataset developed in this study can be used to enhance user immersion in VR environments and as a foundational resource for advancing emotion recognition and authentication technologies in fields such as education, therapy, and entertainment. This dataset offers new research opportunities for non-invasive continuous authentication and emotion recognition in VR environments, and it is expected to significantly contribute to the future development of related technologies.

The Design and Usability of Virtual Reality Content Using Eyes for Students with Severe Disabilities

The purposes of this study were to develop the design of virtual reality content using eyes and to investigate the usability of the virtual reality content for nonverbal students with severe disabilities. A 360-degree video in which a narrator introduces scientific activities (momentum, earthquake, gravity, robot dance) and experiences them at Chungnam Science Education Center was produced. Next, the virtual reality content for the video were designed for users to manipulate image buttons with their eyes so that users can select and experience images. Users can select and view images of scientific activity by staring and blinking using the HTC-VIVE Pro Eye, which combines eye tracking technology. The virtual reality content using eyes were designed and developed in accordance with the guidelines for supporting self-determination of students with severe disabilities. In order to investigate the usability of the VR content, 37 special education teachers who had special teacher certificates and had experience in teaching students with severe disabilities answered questions after using the VR content. The teachers evaluated as "neutral" in some usability evaluation categories (e.g., Self-determination, Ease of manipulation, Ease of Understanding, Effectiveness, Efficiency of class preparation, Efficiency in teaching, Appropriateness of content organization). When developing VR contents for students with severe disabilities, developers can design visual images in order for students with severe disabilities to understand easily and establish user interface in users’ eye conditions and operational capabilities. VR contents can be designed in relation to subject areas so that teachers can use them as educational materials.

High-fidelity color characterization in virtual reality across head mounted displays, game engines, and materials.

We present a comprehensive colorimetric analysis of three head mounted displays (HMDs) - HTC Vive Pro Eye, Pimax 8K X DMAS, and Varjo Aero - focusing on their color calibration and uniformity across different game engines (Unity and Unreal) and for different materials/shaders. We developed a robust methodology combining hardware and software tools, including spectroradiometry and imaging colorimetry, to characterize and calibrate these HMDs for accurate color reproduction. The study showcases substantial advancements in colorimetric accuracy, with a reduction in the average deltaE00 of 90% or more across all tested HMDs and conditions. This level of color reproduction quality is below human discrimination thresholds, ensuring that any color inaccuracies remain imperceptible to the human eye. We also identified key areas for improvement, particularly in display uniformity, which could impact peripheral color reproduction. By making our tools and code publicly available, this study aims to facilitate future research and development in virtual reality (VR) technology, emphasizing the importance of color fidelity in virtual environments. The new insight enabled by our work is the extension and application of a traditional calibration method to currently available HMDs.

Gaze-based attention network analysis in a virtual reality classroom

This article provides a step-by-step guideline for measuring and analyzing visual attention in 3D virtual reality (VR) environments based on eye-tracking data. We propose a solution to the challenges of obtaining relevant eye-tracking information in a dynamic 3D virtual environment and calculating interpretable indicators of learning and social behavior. With a method called "gaze-ray casting," we simulated 3D-gaze movements to obtain information about the gazed objects. This information was used to create graphical models of visual attention, establishing attention networks. These networks represented participants' gaze transitions between different entities in the VR environment over time. Measures of centrality, distribution, and interconnectedness of the networks were calculated to describe the network structure. The measures, derived from graph theory, allowed for statistical inference testing and the interpretation of participants' visual attention in 3D VR environments. Our method provides useful insights when analyzing students’ learning in a VR classroom, as reported in a corresponding evaluation article with N = 274 participants.•Guidelines on implementing gaze-ray casting in VR using the Unreal Engine and the HTC VIVE Pro Eye.•Creating gaze-based attention networks and analyzing their network structure.•Implementation tutorials and the Open Source software code are provided via OSF: https://osf.io/pxjrc/?view_only=1b6da45eb93e4f9eb7a138697b941198.

Application of immersive virtual reality for assessing chronic neglect in individuals with stroke: the immersive virtual road-crossing task

ABSTRACT Background Neglect can be a long-term consequence of chronic stroke that can impede an individual’s ability to perform daily activities, but chronic and discrete forms can be difficult to detect. We developed and evaluated the “immersive virtual road-crossing task” (iVRoad) to identify and quantify discrete neglect symptoms in chronic stroke patients. Method The iVRoad task requires crossing virtual intersections and placing a letter in a mailbox placed either on the left or right. We tested three groups using the HTC Vive Pro Eye: (1) chronic right hemisphere stroke patients with (N = 20) and (2) without (N = 20) chronic left-sided neglect, and (3) age and gender-matched healthy controls (N = 20). We analyzed temporal parameters, errors, and head rotation to identify group-specific patterns, and applied questionnaires to measure self-assessed pedestrian behavior and usability. Results Overall, the task was well-tolerated by all participants with fewer cybersickness-induced symptoms after the VR exposure than before. Reaction time, left-sided errors, and lateral head movements for traffic from left most clearly distinguished between groups. Neglect patients committed more dangerous crossings, but their self-rated pedestrian behavior did not differ from that of stroke patients without neglect. This demonstrates their reduced awareness of the risks in everyday life and highlights the clinical relevance of the task. Conclusions Our findings suggest that a virtual road crossing task, such as iVRoad, has the potential to identify subtle symptoms of neglect by providing virtual scenarios that more closely resemble the demands and challenges of everyday life. iVRoad is an immersive, naturalistic virtual reality task that can measure clinically relevant behavioral variance and identify discrete neglect symptoms.

Pilot Study Assessing the Safety and Acceptance of a Novel Virtual Reality System to Improve Visual Function

ABSTRACT Purpose To assess the feasibility of the clinical use of a novel Virtual Reality (VR) training software designed to be used for active vision therapy in amblyopic patients by determining its preliminary safety and acceptance on the visual function of healthy adults. Methods Pilot study enrolling 10 individuals (3 men, 7 women, mean age: 31.8 ± 6.5 years) with a best-corrected visual acuity (BCVA) of ≥ .90 (decimal) in both eyes were evaluated before and after 20 minutes of exposure to the NEIVATECH VR system using the HTC Vive Pro Eye head mounted display. Visual function assessment included near (40 cm) and distance (6 m) cover test (CT), stereopsis, binocular accommodative facility (BAF), near point of convergence (NPC), near point of accommodation (NPA), accommodative-convergence over accommodation (AC/A) ratio and positive and negative fusional vergences. Safety was assessed using the VR Sickness Questionnaire (VRSQ) and acceptance using the Technology Acceptance Model ;(TAM). Changes in all these variables after VR exposure were analyzed. Results Short-term exposure to the NEIVATECH VR system only induced statistically significant changes in distance phoria (p = .016), but these changes were not clinically relevant. No significant changes were observed in VRSQ oculo-motricity and disorientation scores after exposure (p = .197 and .317, respectively). TAM scores showed a good acceptance of the system in terms of perceived enjoyment and perceived ease of use, although some concerns were raised in relation to the intention-to-use domain. Conclusion Exposure to the NEIVATECH VR system does not seem to adversely affect the visual function in healthy adults and its safety and acceptance profile seems to be adequate for supporting its potential use in other populations, such as amblyopic patients.

A Multi-user Cross-platform hands-on virtual lab within the Metaverse – the case of machining training

Distance learning has become a popular learning channel today. However, while various distance learning tools are available, most of them only support a single platform, offer only the trainer’s perspective, and do not facilitate student-instructor interaction. As a result, distance learning systems tend to be inflexible and less effective. To address the limitations of existing distance learning systems, this study developed a cross-platform hands-on virtual lab within the Metaverse that enables multi-user participation and interaction for distance education. Four platforms, HTC VIVE Pro, Microsoft HoloLens 2, PC, and Android smartphone, are supported. The virtual lab allows trainers to demonstrate operation steps and engage with multiple trainees simultaneously. Meanwhile, trainees have the opportunity to practice their operational skills on their virtual machines within the Metaverse, utilizing their preferred platforms. Additionally, participants can explore the virtual environment and interact with each other by moving around within the virtual space, similar to a physical lab setting. The user test compares the levels of presence and usability in the hands-on virtual lab across different platforms, providing insights into the challenges associated with each platform within the Metaverse for training purposes. Furthermore, the results of the user test highlight the promising potential of the architecture due to its flexibility and adaptability.

Immersive virtual reality for complex skills training: content analysis of experienced challenges

This study aimed to examine the challenges that adult participants experienced in immersive virtual reality (I-VR). Practitioners have indicated that some challenges persist from trainee to trainee and scholars have called for the design and development of virtual reality (VR) applications based on learning theories. Thus, we examined challenges immersed learners experienced during self-discovery of game mechanics and assembly task within an early-development I-VR program. We clarified the immersive learning phenomenon by studying the self-reported problem statements from 168 university students and staff. They used an HTC Vive Pro Eye device and a custom-built software. Through an iterative content analysis of post-survey and video-stimulated recall interviews, we retrieved 481 problem statements from the participants. As a result, we derived and detailed 89 challenges, 22 component features, 11 components, and 5 principal factors of immersive learning. The most cited components that the participants found challenging were the use of controllers and functions, reciprocal software interaction, spatial and navigational constraints, relevance realisation, and learner capabilities. Closer inspection of the quantified data revealed that the participants without digital gaming experience reported relatively more hardware-related problem statements. The findings regarding the constraints of immersive learning helped clarify the various actants involved in immersive learning. In this paper, we provide a design implication summary for VR application developers. Further research on theory-based development and design implications in various immersive training settings is needed.

Can head sway patterns differentiate between patients with Meniere's disease vs. peripheral vestibular hypofunction?

Meniere's disease (MD) is defined by episodic vertigo, unilateral sensorineural hearing loss and fluctuating aural symptoms. Due to the variable clinical presentation, objective tests of MD may have significant diagnostic utility. Head kinematics derived from a head-mounted display (HMD) have demonstrated to be sensitive to vestibular dysfunction. The purpose of this pilot study was to investigate whether head sway can differentiate between patients with MD, vestibular hypofunction (VH) and healthy controls. 80 adults (30 healthy controls, 32 with VH, and 18 with MD) were recruited from a tertiary vestibular clinic. All underwent a postural control assessment using the HTC Vive Pro Eye HMD that recorded head sway in the anterior-posterior (AP), medio-lateral (ML), pitch, yaw and roll direction. Participants were tested with 2 levels of visual load: a static versus oscillating star display. Each scene lasted 60 s and was repeated twice. Sway in each direction was quantified using root mean square velocity (VRMS) for the first 20 s and full 60 s of each scene. Static visual: participants with VH showed significantly larger head VRMS than controls in the AP (60 s and 20 s) and pitch (20 s) directions. Dynamic visual: participants with VH showed significantly larger head VRMS than controls all directions for both the 60 and 20 s analysis. Participants with MD did not differ significantly from the control or the VH group. While limited in numbers, Patients with MD had a high variability in head sway in all directions, and their average head sway was between controls and those with VH. A larger sample as well as patients with worse symptoms at time of testing could elucidate whether head sway via HMD could become a viable test in this population. A similar finding between 20- and 60-s scene and the full portability of the system with an in-clinic testing setup could help these future endeavors. Head sway derived from HMD is sensitive to VH and can be clinically useful as an outcome measure to evaluate sensory integration for postural control.

Architectonic Design Supported by Visual Environmental Simulation—A Comparison of Displays and Formats

Visual environmental simulations are fundamental in understanding the relationship between the built environment and psychological perception. The remarkable evolution of virtual immersion displays over recent years has provided a series of advantages to the architectural discipline, one of which is that non-specialists now have the potential to better understand architectural spaces. This work aimed to analyse the adequacy of the main displays and formats currently used in environmental simulations. As the objective was twofold, two experimental studies were carried out (with a sample of 100 participants). The studies evaluated users’ responses to different environmental representations of two environments, using differential semantic scales to measure key underlying factors (utility, credibility, realism, accuracy, abstraction). The first study examined simulation displays: a PC, an HTC Vive Pro 2 head-mounted display, a PowerWall Screen and a CAVE. In the second, formats were analysed: normal image, 360° image, video and 360° video. The results of this work revealed that users perceived the space differently depending on the representation displays and formats used. Such comparisons of these new means of representing architectural spaces can be helpful to researchers, architects and urban planning professionals and might provoke debate in, and be extrapolated into, the design field.

Spatially dependent veiling glare degrades image quality in medical extended reality

Spatially dependent veiling glare in medical extended reality (MXR) degrades image quality and needs to be characterized across technologies. Measurement methods of veiling glare on virtual and augmented reality (VR and AR) head-mounted displays (HMDs) have not been established in regulatory evaluation of MXR devices or display measurement standards. We describe an experimental bench setup enabling highly flexible translations and rotations of the light measuring device in five degrees of freedom within the eye box. Glare point spread functions (PSFs) of multiple VR and AR headsets are extracted and compared across the display field of view (FOV) in dark environment. At the center, the evaluated VR headsets (HTC VIVE Pro and VIVE Pro 2) demonstrate reduced long-range glare compared to the tested AR HMDs (Microsoft HoloLens 2 and Epson Moverio BT-300). The measured PSFs at multiple locations are spatially invariant for the HoloLens 2. However, veiling glare on the evaluated VR HMDs substantially increases toward the periphery of the FOV primarily due to optical aberration. For VR devices in medical use, increased peripheral glare can lead to image quality degradation and poor visualization of anatomical details.

Analysis of Eye and Head Tracking Movements During a Puck-Hitting Task in Ice Hockey Players, Compared to Wrestlers and Controls.

The study of eye-movement strategies of athletes of various disciplines and skill levels is highly significant for sports psychology, since the results can be used in training to improve performance. Such studies are extremely scarce for ice hockey. To determine successful eye-movement strategies for ice hockey players compared to wrestlers and controls (non-athletes) during puck-hitting tasks of various degrees of difficulty, using virtual reality. The study involved 31 participants (male), including 13 ice hockey players (age 20 ± 2.5), 9 wrestlers (age 19 ± 1.9), and 9 controls (age 19 ± 1.3). We used a pre-developed VR-PACE technology that simulates an ice rink in virtual reality (VR). The task was to hit pucks. VR was presented via the HTC Vive Pro Eye with a built-in eye tracker (100 Hz). We analyzed the parameter that reflected the share of puck presence in one of selected retina areas (0-5°, 5-10°, 10-15°, 15-25°, 25-35°) of the left and right eyes and the head. Ice hockey players exhibited longer puck-tracking using both the near periphery (5-15°) and central retinal area (0-5°). Puck speed had minimal impact on eye-movement strategies, and the visual focus on these areas remained consistent regardless of task type. For both wrestlers and controls, visual fixations in the central retinal area increased when tracking the puck without a motor response, likely leading to higher energy consumption and sensory fatigue. The optimal eye-movement strategy involves parafoveal tracking in the near periphery (5-15°) and partial foveal tracking (0-5°), allowing for better object information retention and efficient puck trajectory tracking with reduced energy expenditure.

Extended Reality Telemedicine Collaboration System Using Patient Avatar Based on 3D Body Pose Estimation.

Recent advances in extended reality (XR) technology have opened the possibility of significantly improving telemedicine systems. This is primarily achieved by transferring 3D information about patient state, which is utilized to create more immersive experiences on VR/AR headsets. In this paper, we propose an XR-based telemedicine collaboration system in which the patient is represented as a 3D avatar in an XR space shared by local and remote clinicians. The proposed system consists of an AR client application running on Microsoft HoloLens 2 used by a local clinician, a VR client application running on the HTC vive Pro used by a remote clinician, and a backend part running on the server. The patient is captured by a camera on the AR side, and the 3D body pose estimation is performed on frames from this camera stream to form a 3D patient avatar. Additionally, the AR and VR sides can interact with the patient avatar via virtual hands, and annotations can be performed on a 3D model. The main contribution of our work is the use of 3D body pose estimation for the creation of a 3D patient avatar. In this way, 3D body reconstruction using depth cameras is avoided, which reduces system complexity and hardware and network resources. Another contribution is the novel architecture of the proposed system, where audio and video streaming are realized using WebRTC protocol. The performance evaluation showed that the proposed system ensures high frame rates for both AR and VR client applications, while the processing latency remains at an acceptable level.

The Use of Virtual Screens in Perceptual Studies on Electric Light: A Comparison Between 2D, Panoramic, Cardboard, and Head-Mounted Displays

ABSTRACT This paper introduces the results of an experiment on lighting perception and reading speed where data collected in a physical space is compared to that collected in representations of that space simulated through a web survey and three VR Head-Mounted Displays (HMDs). The results build on an experiment, where perceptions of electric lighting conditions in a physical space were compared to those of the same space simulated in a single VR HMD (HTC Vive Pro). This study extends the previous scope to compare three VR HMD groups (cardboard VR, Oculus Quest 2, and HTC Vive Pro) to ratings of the same lighting conditions viewed through 2D-perspective or immersive panoramic images displayed in a web survey. Past research has compared perceptions of daylight and electric lighting scenes between physical and VR environments, but there is a lack of research that compares ratings of lighting conditions between VR HMDs and to other conventional image formats. For the VR HMD groups, the results revealed significant differences in reading speed between the physical space and cardboard VR HMD groups. An analysis of our subjective ratings showed all three VR HMDs producing fewer significantly different ratings from the physical space than either of the web groups, but ratings in the panoramic group outperformed the 2D-perspective group. The findings suggest that the type of VR HMD impacts reading speed in virtual lighting scenes and more immersive images produce less error than 2D-perspective images when compared to ratings in a physical space.

Virtual Reality Teleoperation System for Mobile Robot Manipulation

Over the past few years, the industry has experienced significant growth, leading to what is now known as Industry 4.0. This advancement has been characterized by the automation of robots. Industries have embraced mobile robots to enhance efficiency in specific manufacturing tasks, aiming for optimal results and reducing human errors. Moreover, robots can perform tasks in areas inaccessible to humans, such as hard-to-reach zones or hazardous environments. However, the challenge lies in the lack of knowledge about the operation and proper use of the robot. This work presents the development of a teleoperation system using HTC Vive Pro 2 virtual reality goggles. This allows individuals to immerse themselves in a fully virtual environment to become familiar with the operation and control of the KUKA youBot robot. The virtual reality experience is created in Unity, and through this, robot movements are executed, followed by a connection to ROS (Robot Operating System). To prevent potential damage to the real robot, a simulation is conducted in Gazebo, facilitating the understanding of the robot’s operation.

Who is Alyx? A new behavioral biometric dataset for user identification in XR

Introduction: This paper addresses the need for reliable user identification in Extended Reality (XR), focusing on the scarcity of public datasets in this area.Methods: We present a new dataset collected from 71 users who played the game “Half-Life: Alyx” on an HTC Vive Pro for 45 min across two separate sessions. The dataset includes motion and eye-tracking data, along with physiological data from a subset of 31 users. Benchmark performance is established using two state-of-the-art deep learning architectures, Convolutional Neural Networks (CNN) and Gated Recurrent Units (GRU).Results: The best model achieved a mean accuracy of 95% for user identification within 2 min when trained on the first session and tested on the second.Discussion: The dataset is freely available and serves as a resource for future research in XR user identification, thereby addressing a significant gap in the field. Its release aims to facilitate advancements in user identification methods and promote reproducibility in XR research.

The effect of color on short-term memory performance

Introduction. The relevance of the research topic is justified by new trends in modern education, namely, active introduction of innovative technologies and solutions to expand opportunities and improve the quality of education. In the context of educational improvements, the focus is on searching for advanced techniques that can increase the efficiency of cognitive processing of educational information and facilitate memorization of educational material. The study aims to assess the impact that various conditions of color stimulation have on short-term verbal memory indicators (capacity, memorization efficiency, trajectories and successful learning of vocabulary items). Methods. The study involved 53 participants (15 male and 38 female) aged 18-36 years (average age 20.5 years, SD = 3.59). Short-term memory indicators were measured with the help of ten-word memorization technique and a test for memorizing numerical series. The color stimuli of the experiment included the focal colors of the 27 most popular categories among Russian speakers, the coordinates of which were determined in the CIELAB system. To develop and conduct the experiment, the HTC VIVE PRO Eye virtual reality system was used, which allowed demonstrating the experimental stimuli on the helmet screens and simultaneously record eye movements. Results. Memory span averaged 5.6±1.5 and was higher among the participants who saw dark and red-violet hues and lower among those who looked at swamp green and green stimuli. The highest efficiency coefficients for memorizing words were recorded in cases with black (median = 0.87) and dark purple (median = 0.84) color stimulation. When memorizing numerical series, the maximum coefficient was for saturated hues – red, green and fuchsia (median = 0.86). Successful memorization averaged 7.7±1.8 and reached the maximum (10) in cases with exposure to primary basic colors – black, green and blue, as well as non-primary turquoise and crimson. Cluster data analysis showed that memory indicators are most similar for the stimuli that are comparable in their impact on pupil diameter, blinking frequency and dynamics of gaze direction, namely, for light warm hues; dark hues; rich stimuli from the warm part of the color spectrum; light purple hues; red and gray-green hues. Discussion. As in previous studies, certain indicators of short-term memory were higher under the influence of ‘warm’ stimuli compared to ‘cold’ ones, but the present study did not confirm the advantage of achromatic colors compared to chromatic ones. The experiment revealed that changes in memory parameters correlate not only with the tone of color stimuli but also with their other characteristics, primarily with lightness and saturation. Conclusion. The results of this study can be applied used in educational practice to develop effective ways of improving memory performance and of memorizing information using color. Further research should regard possible gender, age and cross-cultural differences.

Development and evaluation of a virtual reality driving test for patients with cirrhosis.

Patients with cirrhosis and especially those with HE may have impaired driving skills and may be prone to car accidents. This proof-of-concept study aimed to develop and evaluate the applicability of a virtual reality (VR)-based driving test in patients with cirrhosis. Additionally, the association between the results in the VR test and car accidents was investigated. A short driving test using a VR head-mounted display (HTC Vive Pro Eye) was developed to simulate five hazardous situations. The patient has to pull the brake when the hazardous situations occur. The time from triggering the event to pressing the brake is recorded as reaction time. Total reaction time (TRT) is defined as the combined reaction time to all 5 events. Car accidents were assessed retrospectively (previous 12 months) and patients were followed prospectively for 6 months. Minimal HE (MHE) was diagnosed using Psychometric Hepatic Encephalopathy Score. In total, 112 outpatients with cirrhosis and 52 controls without cirrhosis were recruited. MHE was detected in 14% (n = 15). Patients with cirrhosis and MHE (5.67s) had higher TRTs compared to patients without MHE (5.02s) and controls without cirrhosis above the age of 50 (4.98s) (MHE vs. no MHE or controls p<0.01, no MHE vs. controls: not significant). Seven patients reported car accidents during the twelve months prior to study inclusion. TRTs were numerically higher in patients with reported car accidents (p = 0.099). When patients who stopped driving for HE-related/cirrhosis-related reasons (n = 14) were added to the group of patients with accidents, then a longer TRT was significantly associated with the modeled outcome in univariable (p<0.01) and multivariable analyses (OR 2.83, p<0.01). Two car accidents occurred during follow-up. Both patients had TRTs above the 90th percentile. The VR driving test is easy and rapid to perform in patients with cirrhosis and could be helpful as a point-of-care tool for predicting car accidents.

Impact of task constraints on a 3D visuomotor tracking task in virtual reality

Objective: The aim of the present study was to evaluate the impact of different task constraints on the participants’ adaptation when performing a 3D visuomotor tracking task in a virtual environment.Methods: Twenty-three voluntary participants were tested with the HTC Vive Pro Eye VR headset in a task that consisted of tracking a virtual target moving in a cube with an effector controlled with the preferred hand. Participants had to perform 120 trials according to three task constraints (i.e., gain, size, and speed), each performed according to four randomized conditions. The target-effector distance and elbow range of movement were measured.Results: The results showed an increase in the distance to the target when the task constraints were the strongest. In addition, a change in movement kinematics was observed, involving an increase in elbow amplitude as task constraints increased. It also appeared that the depth dimension played a major role in task difficulty and elbow amplitude and coupling in the tracking task.Conclusion: This research is an essential step towards characterizing interactions with a 3D virtual environment and showing how virtual constraints can facilitate arm’s involvement in the depth dimension.

ENERGYSIM: techniques for advancing building energy education through immersive virtual reality (VR) simulation

An important practice for reducing the effects of global warming is the design and construction of energy-efficient buildings. In design education, the full comprehension of thermal behavior in buildings based on their geometry and material composition is required. The complexity of energy simulation principles, vis-a-vis the number of elements that impact the energy loads, their linkages, and their relationships to one another all combine to make this a challenging subject to absorb. Virtual Reality (VR) provides an immersive way to learn the concepts of building energy responses; however, the development of VR applications for education is difficult due to the knowledge, skill, and performance resource-related gaps. Unoptimized VR applications can adversely impact learning if user experiences are broken due to performance lags. This research, therefore, explores VR as a teaching tool for building energy education while showcasing the development process toward a visually accurate simulation and performant application. We developed EnergySIM; a multi-user VR building energy simulation prototype of the famous Farnsworth House. Using this prototype, we document rigorously tested development workflows for improved VR game performance, high visual fidelity, and user interaction, the three key factors which positively contribute to user knowledge retention. The study combines menu-driven interaction, virtual exploration, and miniature model manipulation approaches with the aim of testing user understanding and knowledge retention. Highlighted results provide reduced barriers of entry for educators towards developing higher quality educational VR applications. EnergySIM showcases pre-simulated building exterior surface heatmaps response from four seasons (winter, summer, fall, and spring) alongside an all-year-round sun-hour scenario. Four different material pre-simulated scenarios (single glazing, double glazing, concrete, and wood) for interior atmospheric temperature mapping are also explored. Preferred interaction methods are documented by allowing users’ visual appraisal of alternative building materials based on insulation capacity or resistance to heat flow (R-value). The significance of this work lies in its potential to revolutionize how students, designers, and instructors approach building energy education in today’s world. EnergySIM provides a hands-on and visually engaging learning experience towards the enhancement of knowledge retention and understanding. It pushes the boundaries of development for visual fidelity using geometry/mesh modeling input from various software into game engines and optimizing game performance using the HTC Vive Pro Eye and Meta Quest Pro headsets.