3D Virtual Reality Environment Research Articles

Combining 360° Video and Camera Mapping for Virtual Reality: An Innovative Solution

Changing the framework of traditional video with limited viewing angles, 360° photo/video provides an immersive viewing experience. 360° video is one of the applications and the most important feature of virtual reality for immersion and the feeling of being in another space. However, when viewing 360° videos with a head-mounted display, the viewer feels like a fixed rotatable camera, and the viewer's movement does not change the viewing angle of the object, which greatly reduces the spatial immersion required for virtual reality. Therefore, we propose a solution that maintains high-quality graphics and low hardware demands and supports 6DoF head-mounted displays. Through the camera mapping function in the 3D animation software, 360° surround video is projected into a 3D sphere to create a simple 3D object that corresponds to the shape of the object in the image. With pristine video quality and a realistic 3D spatial perspective, it provides better virtual reality immersion than 360° video and is not as complex as a full 3D virtual reality environment. In the future, 3D scanning and photogrammetry can be integrated to reconstruct a more easily applied 3D virtual reality environment.

Metaverse Engagement Metrics, Workplace Monitoring and Tracking Systems, Behavioral Predictive Analytics in Immersive 3D Virtual Reality Environments

Metaverse Engagement Metrics, Workplace Monitoring and Tracking Systems, Behavioral Predictive Analytics in Immersive 3D Virtual Reality Environments

Emotional arousal in 2D versus 3D virtual reality environments

Previous studies have suggested that virtual reality (VR) can elicit emotions in different visual modes using 2D or 3D headsets. However, the effects on emotional arousal by using these two visual modes have not been comprehensively investigated, and the underlying neural mechanisms are not yet clear. This paper presents a cognitive psychological experiment that was conducted to analyze how these two visual modes impact emotional arousal. Forty volunteers were recruited and were randomly assigned to two groups. They were asked to watch a series of positive, neutral and negative short VR videos in 2D and 3D. Multichannel electroencephalograms (EEG) and skin conductance responses (SCR) were recorded simultaneously during their participation. The results indicated that emotional stimulation was more intense in the 3D environment due to the improved perception of the environment; greater emotional arousal was generated; and higher beta (21-30 Hz) EEG power was identified in 3D than in 2D. We also found that both hemispheres were involved in stereo vision processing and that brain lateralization existed in the processing.

Does the Size-Arrival Effect Occur With an Active Collision-Avoidance Task in an Immersive 3D Virtual Reality Environment?

Determine whether the size-arrival effect (SAE) occurs with immersive, 3D visual experiences and active collision-avoidance responses. When a small near object and a large far object approach the observer at the same speeds, the large object appears to arrive before the small object, known as the size-arrival effect (SAE), which may contribute to crashes between motorcycles and cars. Prior studies of the SAE were limited because they used two dimensional displays and asked participants to make passive judgments. Participants viewed approaching objects using a virtual reality (VR) headset. In an active task, participants ducked before the object hit them. In a passive prediction-motion (PM) judgment, the approaching object disappeared, and participants pressed a button when they thought the object would hit them. In a passive relative TTC judgment, participants reported which of two approaching objects would reach them first. The SAE occurred with the PM and relative TTC tasks but not with the ducking task. The SAE can occur in immersive 3D environments but is limited by the nature of the task and display. Certain traffic situations may be more prone to the SAE and have higher risk for collisions. For example, in left-turn scenarios (e.g., see Levulis, 2018), drivers make passive judgments when oncoming vehicles are far and optical expansion is slow, and binocular disparity putatively is ineffective. Collision-avoidance warning systems may be needed more in such scenarios than when vehicles are near and drivers' judgments of TTC may be more accurate (DeLucia, 2008).

Haptic-enabled collaborative learning in virtual reality for schools

This paper reports on a study which designed and developed a multi-fingered haptic interface in conjunction with a three-dimensional (3D) virtual model of a section of the cell membrane in order to enable students to work collaboratively to learn cell biology. Furthermore, the study investigated whether the addition of haptic feedback to the 3D virtual reality (VR) simulation affected learning of key concepts in nanoscale cell biology for students aged 12 to 13. The haptic interface was designed so that the haptic feedback could be turned on or switched off. Students (N = 64), in two secondary schools, worked in pairs, on activities designed to support learning of specific difficult concepts. Findings from observation of the activities and interviews revealed that students believed that being immersed in the 3D VR environment and being able to feel structures and movements within the model and work collaboratively assisted their learning. More specifically, the pilot/co-pilot model that we developed was successful for enabling collaborative learning and reducing the isolating effects of immersion with a 3D headset. Results of pre and post-tests of conceptual knowledge showed significant knowledge gains but addition of haptic feedback did not affect the knowledge gains significantly. The study enabled identification of important issues to consider when designing and using haptic-enabled 3D VR environments for collaborative learning.

Platform is not destiny: Embodied learning effects comparing 2D desktop to 3D virtual reality STEM experiences

AbstractExperiences in immersive 3D virtual reality (VR) are more presence‐inducing, and so it may be tempting to claim that content will be learned better in VR. This randomized control trial study on natural selection challenges that assumption. This study answers the question of whether learning STEM in an immersive 3D VR environment is always superior to learning via a 2D monitor (PC). This is a 2 × 2 × 3 design. The first factor is platform immersivity (low = 2D PC, or high = 3D VR), the second factor is level of embodiment (lower = watching playback video, or higher = using mouse/controller to agentically manipulate content), and the third factor is test time (pretest, posttest, and follow‐up). There was a significant main effect for embodiment, the high embodied and agentic groups learned the most. There was not a main effect for platform, because the participants in the low embodied VR group performed significantly worse than the three other groups. Although, the one high embodied, VR group learned and retained the most knowledge. A path‐analysis revealed that the effect of platform was significantly mediated by presence, agency, and engagement. The smaller learning gain in the low embodied VR condition suggests that participants come to the immersive VR experience with expectations about agency and control of the virtual content, and when those expectations were not met, the disconnect was deleterious for learning. More agentic and interactive control of manipulable virtual content is encouraged. Design is critical, and platform is not destiny.

Interoceptive Dramaturgies of ‘Surrogate-selves’ in Performative VR Experiences

Through a close analysis of interoceptive dramaturgies, and by drawing on neuroscientific and cognitive definitions of interoception, this article will explore what is at stake in employing interoceptive oriented design in performative virtual reality (VR) environments and the ethical implications of manipulating corporeal experiences through these perceptual modalities. I examine two contrasting performative VR experiences: a one-to-one interactive virtual reality performance—Goodnight, Sleep Tight (2017) by ZU-UK—and Half-Life Alyx (2020)—a VR game by Valve. The first case study is part one-to-one performance and 360-degree VR video inviting the participant to experience a dream through the eyes of a child. It is a gentle, poetic and subtly constructed VR experience that draws the participant into the world of childhood memories. The second case study is a survival horror videogame set in a three-dimensional (3D) VR environment replete with violence, gore and invasive interactivity. Despite the contrasts between the two case studies I contend, following Liam Jarvis’s (2019) insight, that in both cases players/participants are construed and perform as ‘surrogate selves’ of their respective avatars. I also argue that their experiences are to a great extent conditioned through interoceptive aesthetics.

EEG Cortical Activities and Networks Altered by Watching 2D/3D Virtual Reality Videos

Abstract. Virtual reality (VR), which can represent real-life events and situations, is being increasingly applied to many fields, such as education, entertainment, and medical rehabilitation. Correspondingly, the neural information processing of VR has attracted attention. However, the underlying neural mechanisms of VR environments have not yet been fully revealed. The purpose of this study was to examine the possible differences in brain activities and networks between the less immersive 2D and the fully immersive 3D VR environments. 3D VR videos and the same 2D scenes were presented to the participants and the scalp electroencephalogram (EEG) was recorded, respectively. Power spectral density (PSD) and the functional connectivity of these EEG signals were analyzed. The results showed that 3D VR videos significantly enhanced the PSD of θ rhythm (4–7 Hz) in the frontal lobe; decreased the PSD of α rhythm (8–13 Hz) in the parietal and the occipital lobes; increased the PSD of β rhythm (14–30 Hz) in the frontal, the parietal, the temporal, and the occipital lobes, relative to 2D VR watching. Furthermore, 3D versus 2D VR-induced alterations in the patterns of brain networks were similar to the patterns of PSD. Specifically, for the θ rhythm, 3D VR significantly enhanced the frontal and the temporal brain functional connectivity; for the α rhythm, 3D VR increased the parietal and the occipital networks; for the β rhythm, 3D VR remarkably increased the frontal, the occipital, the frontal-temporal and the frontal-occipital brain functional connectivity, relative to 2D VR. These significant differences between 3D and 2D VR video-watching suggest that the neural information processing of cortical activities and networks is correlated to the degree of immersion. The present results, collected with previous researches, implicate that some visual-related information processes, such as visual attention, visual perception, and visual immersion are more robust in 3D VR environments.

The Smart Aging Platform for Assessing Early Phases of Cognitive Impairment in Patients With Neurodegenerative Diseases.

Background: Smart Aging is a serious game (SG) platform that generates a 3D virtual reality environment in which users perform a set of screening tasks designed to allow evaluation of global cognition. Each task replicates activities of daily living performed in a familiar environment. The main goal of the present study was to ascertain whether Smart Aging could differentiate between different types and levels of cognitive impairment in patients with neurodegenerative disease.Methods: Ninety-one subjects (mean age = 70.29 ± 7.70 years)—healthy older adults (HCs, n = 23), patients with single-domain amnesic mild cognitive impairment (aMCI, n = 23), patients with single-domain executive Parkinson's disease MCI (PD-MCI, n = 20), and patients with mild Alzheimer's disease (mild AD, n = 25)—were enrolled in the study. All participants underwent cognitive evaluations performed using both traditional neuropsychological assessment tools, including the Mini-Mental State Examination (MMSE), Montreal Overall Cognitive Assessment (MoCA), and the Smart Aging platform. We analyzed global scores on Smart Aging indices (i.e., accuracy, time, distance) as well as the Smart Aging total score, looking for differences between the four groups.Results: The findings revealed significant between-group differences in all the Smart Aging indices: accuracy (p < 0.001), time (p < 0.001), distance (p < 0.001), and total Smart Aging score (p < 0.001). The HCs outperformed the mild AD, aMCI, and PD-MCI patients in terms of accuracy, time, distance, and Smart Aging total score. In addition, the mild AD group was outperformed both by the HCs and by the aMCI and PD-MCI patients on accuracy and distance. No significant differences were found between aMCI and PD-MCI patients. Finally, the Smart Aging scores significantly correlated with the results of the neuropsychological assessments used.Conclusion: These findings, although preliminary due to the small sample size, suggest the validity of Smart Aging as a screening tool for the detection of cognitive impairment in patients with neurodegenerative diseases.

Adaptive Rehabilitation Bots in Serious Games.

In recent years, we have witnessed a growing adoption of serious games in telerehabilitation by taking advantage of advanced multimedia technologies such as motion capture and virtual reality devices. Current serious game solutions for telerehabilitation suffer form lack of personalization and adaptiveness to patients’ needs and performance. This paper introduces “RehaBot”, a framework for adaptive generation of personalized serious games in the context of remote rehabilitation, using 3D motion tracking and virtual reality environments. A personalized and versatile gaming platform with embedded virtual assistants, called “Rehab bots”, is created. Utilizing these rehab bots, all workout session scenes will include a guide with various sets of motions to direct patients towards performing the prescribed exercises correctly. Furthermore, the rehab bots employ a robust technique to adjust the workout difficulty level in real-time to match the patients’ performance. This technique correlates and matches the patterns of the precalculated motions with patients’ motions to produce a highly engaging gamified workout experience. Moreover, multimodal insights are passed to the users pointing out the joints that did not perform as anticipated along with suggestions to improve the current performance. A clinical study was conducted on patients dealing with chronic neck pain to prove the usability and effectiveness of our adjunctive online physiotherapy solution. Ten participants used the serious gaming platform, while four participants performed the traditional procedure with an active program for neck pain relief, for two weeks (10 min, 10 sessions/2 weeks). Feasibility and user experience measures were collected, and the results of experiments show that patients found our game-based adaptive solution engaging and effective, and most of them could achieve high accuracy in performing the personalized prescribed therapies.

Does the Size-Arrival Effect Occur with an Active Collision- Avoidance Task in an Immersive 3D Virtual Reality Environment?

ObjectiveDetermine whether the size-arrival effect (SAE) occurs with immersive, 3D visual experiences and active collision-avoidance responses.BackgroundWhen a small near object and a large far obj...

A Novel Method to Understand Neural Oscillations During Full-Body Reaching: A Combined EEG and 3D Virtual Reality Study.

Virtual reality (VR) can be used to create environments that are not possible in the real-world. Producing movements in VR holds enormous promise for rehabilitation and offers a platform from which to understand the neural control of movement. However, no study has examined the impact of a 3D fully immersive head-mounted display (HMD) VR system on the integrity of neural data. We assessed the quality of 64-channel EEG data with and without HMD VR during rest and during a full-body reaching task. We compared resting EEG while subjects completed three conditions: No HMD (EEG-only), HMD powered off (VR-off), and HMD powered on (VR-on). Within the same session, EEG were collected while subjects completed full-body reaching movements in two conditions (EEG-only, VR-on). During rest, no significant differences in data quality and power spectrum were observed between EEG-only, VR-off, and VR-on conditions. During reaching movements, the proportion of components attributed to the brain was greater in the EEG-only condition compared to the VR-on condition. Despite this difference, neural oscillations in source space were not significantly different between conditions, with both conditions associated with decreases in alpha and beta power in sensorimotor cortex during movements. Our findings demonstrate that the integrity of EEG data can be maintained while individuals execute full-body reaching movements within an immersive 3D VR environment. Clinical impact: Integrating VR and EEG is a viable approach to understanding the cortical processes of movement. Simultaneously recording movement and brain activity in combination with VR provides the foundation for neurobiologically informed rehabilitation therapies.

Complexity of postural sway affects affordance perception of reachability in virtual reality.

Visual perception of whether an object is within reach while standing in different postures was investigated. Participants viewed a three-dimensional (3D) virtual reality (VR) environment with a stimulus object (red ball) placed at different egocentric distances. Participants reported whether the object was reachable while in a standard pose as well as in two separate active balance poses (yoga tree pose and toe-to-heel pose). Feedback on accuracy was not provided, and participants were not allowed to attempt to reach. Response time, affordance judgements (reachable and not reachable), and head movements were recorded on each trial. Consistent with recent research on perception of reaching ability, the perceived boundary occurred at approximately 120% of arm length, indicating overestimation of perceived reaching ability. Response times increased with distance, and were shortest for the most difficult pose-the yoga tree pose. Head movement amplitude increased with increases in balance demands. Unexpectedly, the coefficient of variation was comparable in the two active balance poses, and was more extreme in the standard control pose for the shortest and longest distances. More complex descriptors of postural sway (i.e., effort-to-compress) were predictive of perception while in the tree pose and the toe-to-heel pose, as compared with control stance. This demonstrates that standard measures of central tendency are not sufficient for describing multiscale interactions of postural dynamics in functional tasks.

Extending Fitts’ law in three-dimensional virtual environments with current low-cost virtual reality technology

Abstract Virtual reality (VR) interfaces require users to perform three-dimensional reaching and pointing movements to interact with objects positioned within the user's arm's reach. However, there has been limited work that has evaluated the applicability of established models of human motor control to model performance of these tasks in 3D virtual reality environments using current low-cost technologies. In this study, a 3D discrete pointing task using the Oculus Rift system was used to explore potential influences on movement in VR and to account for these influences in a new formulation of Fitts’ law. Target size and distance from the starting point of movement were systematically varied to generate a broad range of index of difficulty (ID) values. Target locations were specified using a spherical coordinate system in which inclination angle corresponded to the pitch of the movement axis with respect to the starting point of movements and azimuth angle corresponded to the roll of the movement axis with respect to the horizontal plane. In line with previous work, we observed that target size, radial distance, and inclination angle had a significant effect on movement time. The effect of inclination angle varied with target size, which suggests that target size affected depth estimation. Significant target characteristics and interaction effects were used to develop an extended Fitts’ law model, which accounted for 64.5% of the variation in movement times. Comparisons to other Fitts’ law models revealed that models accounting for the effects of target depth improved predictive power relative to the traditional Fitts’ law formulation. Together, these findings support the value of extending Fitts’ law models to account for domain-specific constraints in VR environments. We discuss these results in the context of previous work examining HMD display deficiencies and discrete 3D pointing tasks, and suggest several directions for future work.

How 3D Virtual Reality Stores Can Shape Consumer Purchase Decisions: The Roles of Informativeness and Playfulness

The recent rise of consumer virtual reality (VR) hardware raises important questions in the field of online marketing: what makes 3D VR more informative and playful than conventional 2D media such as a still image and a video, and how it affects the online purchase decision-making process. In this study, we mainly focus on three interface features—interactivity, visual–spatial cues, and graphics quality. We explore how each of these three interface features enhances playfulness and informativeness of shopping interface and further influences subsequent product evaluation and purchase intention. The results of the study provide two meaningful insights. First, interactivity and visual–spatial cues significantly enhance perceived informativeness and playfulness; however, the role of graphics quality was found to be more critical for 2D displays than for 3D VR environment. Second, informativeness and playfulness influence the purchase decision-making process in distinct ways. More specifically, a playful interface may enhance consumers' preference for hedonic product benefits (e.g., a stylish and attractive design), whereas informativeness is a more important explanatory variable for subsequent purchase intentions. We discuss the theoretical contribution and managerial insights the research provides for online retailers and designers.

The Embodied Brain of SOVEREIGN2: From Space-Variant Conscious Percepts During Visual Search and Navigation to Learning Invariant Object Categories and Cognitive-Emotional Plans for Acquiring Valued Goals.

This article develops a model of how reactive and planned behaviors interact in real time. Controllers for both animals and animats need reactive mechanisms for exploration, and learned plans to efficiently reach goal objects once an environment becomes familiar. The SOVEREIGN model embodied these capabilities, and was tested in a 3D virtual reality environment. Neural models have characterized important adaptive and intelligent processes that were not included in SOVEREIGN. A major research program is summarized herein by which to consistently incorporate them into an enhanced model called SOVEREIGN2. Key new perceptual, cognitive, cognitive-emotional, and navigational processes require feedback networks which regulate resonant brain states that support conscious experiences of seeing, feeling, and knowing. Also included are computationally complementary processes of the mammalian neocortical What and Where processing streams, and homologous mechanisms for spatial navigation and arm movement control. These include: Unpredictably moving targets are tracked using coordinated smooth pursuit and saccadic movements. Estimates of target and present position are computed in the Where stream, and can activate approach movements. Motion cues can elicit orienting movements to bring new targets into view. Cumulative movement estimates are derived from visual and vestibular cues. Arbitrary navigational routes are incrementally learned as a labeled graph of angles turned and distances traveled between turns. Noisy and incomplete visual sensor data are transformed into representations of visual form and motion. Invariant recognition categories are learned in the What stream. Sequences of invariant object categories are stored in a cognitive working memory, whereas sequences of movement positions and directions are stored in a spatial working memory. Stored sequences trigger learning of cognitive and spatial/motor sequence categories or plans, also called list chunks, which control planned decisions and movements toward valued goal objects. Predictively successful list chunk combinations are selectively enhanced or suppressed via reinforcement learning and incentive motivational learning. Expected vs. unexpected event disconfirmations regulate these enhancement and suppressive processes. Adaptively timed learning enables attention and action to match task constraints. Social cognitive joint attention enables imitation learning of skills by learners who observe teachers from different spatial vantage points.

Design and Evaluation of Visualization Techniques of Off-Screen and Occluded Targets in Virtual Reality Environments.

This research explores the design and evaluation of visualization techniques of targets that reside outside of users' view and/or are occluded by other elements within a virtual reality environment (VE). We first compare four techniques (3DWedge, 3DArrow, 3DMinimap, and Radar) that use different types of visual elements to provide direction and distance information of targets. To give structure to their evaluation, we also develop a framework of four tasks (one for direction and three for distance) and their assessment criteria. The results of the first study show that 3DWedge is the best-performing and most usable technique. However, all techniques, including 3DWedge, have poor performance in dense scenarios with a large number of targets. To improve support in dense scenarios, a fifth technique, 3DWedge+, is developed by using 3DWedge as its foundation and including additional visual elements that are derived from the other three techniques which are shown to be useful. A second study is conducted to evaluate the performance of 3DWedge+ in relation to the other techniques. The results show that both 3DWedge and 3DWedge+ are significantly better in distinguishing user-to-target distance and that 3DWedge+ is particularly suitable for dense scenarios. Based on these results, we provide a set of recommendations for the design of visualization techniques of off-screen and occluded targets in 3D VE.

Study of Lighting and Material Iterations in Full Scale Model Using Virtual Reality and Interactive Architectural Representation

Common architectural visualization is using 3D digital model represented in 2D media such as screen and papers. However, this method has two limitations. First, it does not have a full scale to provide a true perception space. Second, the 2D media to represent the 3D model is static. These limitations diminish the user’s experience and understanding of space. Currently, architectural visualization is starting to utilize technological advances of virtual reality (VR) and interactive game engine. The full-scale projection in VR can minimize the perception gap between the client’s and the designer’s idea. It also has interactivity that enables users to explore the space and perform direct manipulation at some design aspects. This paper observes how VR and interactive technology is used to explore scale, lighting experience, and material iterations. The research was conducted for one semester in 2 courses. In these courses, students worked in groups to create, experience, test, and evaluate their design through 3D VR environment and interactive representation. Afterwards, the VR was tested to common users. We examined the enhancement that users get by experiencing design through VR compared to viewing the architectural visualization on picture and discuss how this could be improved and integrated to curriculum.

GLAM: Glycogen-derived Lactate Absorption Map for visual analysis of dense and sparse surface reconstructions of rodent brain structures on desktop systems and virtual environments

Human brain accounts for about one hundred billion neurons, but they cannot work properly without ultrastructural and metabolic support. For this reason, mammalian brains host another type of cells called “glial cells”, whose role is to maintain proper conditions for efficient neuronal function. One type of glial cell, astrocytes, are involved in particular in the metabolic support of neurons, by feeding them with lactate, one byproduct of glucose metabolism that they can take up from blood vessels, and store it under another form, glycogen granules. These energy-storage molecules, whose morphology resembles to spheres with a diameter ranging 10–80 nanometers roughly, can be easily recognized using electron microscopy, the only technique whose resolution is high enough to resolve them. Understanding and quantifying their distribution is of particular relevance for neuroscientists, in order to understand where and when neurons use energy under this form. To answer this question, we developed a visualization technique, dubbed GLAM (Glycogen-derived Lactate Absorption Map), and customized for the analysis of the interaction of astrocytic glycogen on surrounding neurites in order to formulate hypotheses on the energy absorption mechanisms. The method integrates high-resolution surface reconstruction of neurites, astrocytes, and the energy sources in form of glycogen granules from different automated serial electron microscopy methods, like focused ion beam scanning electron microscopy (FIB-SEM) or serial block face electron microscopy (SBEM), together with an absorption map computed as a radiance transfer mechanism. The resulting visual representation provides an immediate and comprehensible illustration of the areas in which the probability of lactate shuttling is higher. The computed dataset can be then explored and quantified in a 3D space, either using 3D modeling software or virtual reality environments. Domain scientists have evaluated the technique by either using the computed maps for formulating functional hypotheses or for planning sparse reconstructions to avoid excessive occlusion. Furthermore, we conducted a pioneering user study showing that immersive VR setups can ease the investigation of the areas of interest and the analysis of the absorption patterns in the cellular structures.

The Study of the Analysis of a User's Perception of Screen Component for Inducing Emotion in the 3D Virtual Reality Environment

The Study of the Analysis of a User's Perception of Screen Component for Inducing Emotion in the 3D Virtual Reality Environment