Immersive Virtual Reality Environment Research Articles

AdaPIP: Adaptive picture-in-picture guidance for 360° film watching

Abstract360° videos enable viewers to watch freely from different directions but inevitably prevent them from perceiving all the helpful information. To mitigate this problem, picture-in-picture (PIP) guidance was proposed using preview windows to show regions of interest (ROIs) outside the current view range. We identify several drawbacks of this representation and propose a new method for 360° film watching called AdaPIP. AdaPIP enhances traditional PIP by adaptively arranging preview windows with changeable view ranges and sizes. In addition, AdaPIP incorporates the advantage of arrow-based guidance by presenting circular windows with arrows attached to them to help users locate the corresponding ROIs more efficiently. We also adapted AdaPIP and Outside-In to HMD-based immersive virtual reality environments to demonstrate the usability of PIP-guided approaches beyond 2D screens. Comprehensive user experiments on 2D screens, as well as in VR environments, indicate that AdaPIP is superior to alternative methods in terms of visual experiences while maintaining a comparable degree of immersion.

Application of Virtual Reality Technology in English Teaching: Research on the Relationship between Learning Motivation and Learning Effectiveness

Virtual reality technology is an important direction of simulation technology, is a collection of simulation technology and computer graphics, human-computer interface technology, multimedia technology, sensing technology, network technology, and other technologies. It is a challenging cross-technology frontier discipline and research field. Virtual Reality (VR) technology mainly includes simulated environment, perception, natural skills and sensing devices. The simulated environment is a computer-generated, real-time dynamic three-dimensional stereo realistic image. This paper explores the transformative potential of virtual reality (VR) technology in English language teaching, focusing on the dynamic interplay between learning motivation and effectiveness. Through immersive VR environments, educators can cultivate increased engagement and foster language acquisition. A comprehensive literature review demonstrates the widespread adoption of VR technology in various fields and highlights its positive impact on learning outcomes. The proposed methodology advocates the seamless integration of VR technology with innovative pedagogical approaches to optimize English language instruction. By analyzing the relationship between learning motivation and effectiveness, this study sheds light on the nuanced dynamics of VR-enhanced learning in English language education.

Experimental evaluation of the impact of sEMG interfaces in enhancing embodiment of virtual myoelectric prostheses

IntroductionDespite recent technological advances that have led to sophisticated bionic prostheses, attaining embodied solutions still remains a challenge. Recently, the investigation of prosthetic embodiment has become a topic of interest in the research community, which deals with enhancing the perception of artificial limbs as part of users’ own body. Surface electromyography (sEMG) interfaces have emerged as a promising technology for enhancing upper-limb prosthetic control. However, little is known about the impact of these sEMG interfaces on users’ experience regarding embodiment and their interaction with different functional levels.MethodsTo investigate this aspect, a comparison is conducted among sEMG configurations with different number of sensors (4 and 16 channels) and different time delay. We used a regression algorithm to simultaneously control hand closing/opening and forearm pronation/supination in an immersive virtual reality environment. The experimental evaluation includes 24 able-bodied subjects and one prosthesis user. We assess functionality with the Target Achievement Control test, and the sense of embodiment with a metric for the users perception of self-location, together with a standard survey.ResultsAmong the four tested conditions, results proved a higher subjective embodiment when participants used sEMG interfaces employing an increased number of sensors. Regarding functionality, significant improvement over time is observed in the same conditions, independently of the time delay implemented.ConclusionsOur work indicates that a sufficient number of sEMG sensors improves both, functional and subjective embodiment outcomes. This prompts discussion regarding the potential relationship between these two aspects present in bionic integration. Similar embodiment outcomes are observed in the prosthesis user, showing also differences due to the time delay, and demonstrating the influence of sEMG interfaces on the sense of agency.

Intelligent Analysis System for Teaching and Learning Cognitive Engagement Based on Computer Vision in an Immersive Virtual Reality Environment

The 20th National Congress of the Communist Party of China and the 14th Five Year Plan for Education Informatization focus on digital technology and intelligent learning and implement innovation-driven education environment reform. An immersive virtual reality (IVR) environment has both immersive and interactive characteristics, which are an important way of virtual learning and are also one of the important ways in which to promote the development of smart education. Based on the above background, this article proposes an intelligent analysis system for Teaching and Learning Cognitive engagement in an IVR environment based on computer vision. By automatically analyzing the cognitive investment of students in the IVR environment, it is possible to better understand their learning status, provide personalized guidance to improve learning quality, and thereby promote the development of smart education. This system uses Vue (developed by Evan You, located in Wuxi, China) and ECharts (Developed by Baidu, located in Beijing, China) for visual display, and the algorithm uses the Pytorch framework (Developed by Facebook, located in Silicon Valley, CA, USA), YOLOv5 (Developed by Ultralytics, located in Washington, DC, USA), and the CRNN model (Convolutional Recurrent Neural Network) to monitor and analyze the visual attention and behavioral actions of students. Through this system, a more accurate analysis of learners’ cognitive states and personalized teaching support can be provided for the education field, providing certain technical support for the development of smart education.

IVR-fNIRS: studying brain functions in a fully immersive virtual environment.

Immersive virtual reality (iVR) employs head-mounted displays or cave-like environments to create a sensory-rich virtual experience that simulates the physical presence of a user in a digital space. The technology holds immense promise in neuroscience research and therapy. In particular, virtual reality (VR) technologies facilitate the development of diverse tasks and scenarios closely mirroring real-life situations to stimulate the brain within a controlled and secure setting. It also offers a cost-effective solution in providing a similar sense of interaction to users when conventional stimulation methods are limited or unfeasible. Although combining iVR with traditional brain imaging techniques may be difficult due to signal interference or instrumental issues, recent work has proposed the use of functional near infrared spectroscopy (fNIRS) in conjunction with iVR for versatile brain stimulation paradigms and flexible examination of brain responses. We present a comprehensive review of current research studies employing an iVR-fNIRS setup, covering device types, stimulation approaches, data analysis methods, and major scientific findings. The literature demonstrates a high potential for iVR-fNIRS to explore various types of cognitive, behavioral, and motor functions in a fully immersive VR (iVR) environment. Such studies should set a foundation for adaptive iVR programs for both training (e.g., in novel environments) and clinical therapeutics (e.g., pain, motor and sensory disorders and other psychiatric conditions).

Immersive Multimedia Art Design Based on Deep Learning Intelligent Vr Technology

Virtual Reality (VR) based art design offers a revolutionary platform for artists to explore new dimensions of creativity and expression. By immersing users in virtual environments, VR art transcends traditional limitations, enabling the creation of immersive and interactive experiences that defy conventional boundaries. Artists harness VR technology to sculpt three-dimensional forms, paint in volumetric space, and manipulate light and sound in ways previously unimaginable. Moreover, VR facilitates collaboration and audience engagement, allowing viewers to interact with and even become part of the artwork. From virtual exhibitions to immersive installations, VR art opens up endless possibilities for artistic innovation and audience participation. This paper introduces an innovative approach to immersive multimedia art design, leveraging deep learning intelligent Virtual Reality (VR) technology alongside Centralized Data Transmission Classification (CDTC) with the integration of IoT multimedia sensor data. By combining these advanced technologies, artists can create captivating and dynamic multimedia experiences that engage multiple senses and transcend traditional artistic boundaries. Deep learning algorithms analyze and interpret vast amounts of sensory data collected from IoT multimedia sensors, enabling the generation of immersive VR environments that respond intelligently to user interactions. CDTC facilitates efficient data transmission and classification, optimizing the integration of real-time sensor data into the VR experience. Through a series of experiments and simulations, the efficacy of the proposed framework is demonstrated, showcasing its ability to create immersive art installations that dynamically adapt to user input and environmental stimuli. in a simulated VR environment, deep learning algorithms processed IoT sensor data in real-time, resulting in an average classification accuracy of 92% for environmental stimuli recognition. Additionally, CDTC facilitated efficient data transmission, reducing latency by 30% compared to traditional methods.

Behavioral intention, perception and user assessment in an immersive virtual reality environment with CFD simulations

This study explores technology acceptance, perception and user assessment of an immersive virtual reality environment with computational fluid dynamics simulations in engineering education. 57 participants from three different institutions tested the virtual reality application. Partial least squares structural equation modeling and interferential statistics were performed to predict and assess interrelations among constructs. Results show that the learning value, content value, intrinsic motivation and personal innovativeness are underlying factors behind students’ intention to use virtual reality. Pair-wise analysis indicates that users’ perceptions matter and positively affect their attitudes. In addition, the virtual reality application helps students perform significantly better in the post-knowledge test. Findings also highlight that prior experience and interest can affect students’ attitudes and behavioral intentions to accept the virtual reality application in education. Our study can guide lecturers and developers to achieve on-target immersive virtual reality learning environments in higher education.

Thelxinoë: Recognizing Human Emotions Using Pupillometry and Machine Learning

In this study, we present a method for emotion recognition in Virtual Reality (VR) using pupillometry. We analyze pupil diameter responses to both visual and auditory stimuli via a VR headset and focus on extracting key features in the time-domain, frequency-domain, and time-frequency domain from VR-generated data. Our approach utilizes feature selection to identify the most impactful features using Maximum Relevance Minimum Redundancy (mRMR). By applying a Gradient Boosting model, an ensemble learning technique using stacked decision trees, we achieve an accuracy of 98.8% with feature engineering, compared to 84.9% without it. This research contributes significantly to the Thelxinoë framework, aiming to enhance VR experiences by integrating multiple sensor data for realistic and emotionally resonant touch interactions. Our findings open new avenues for developing more immersive and interactive VR environments, paving the way for future advancements in virtual touch technology.

Enhancing balance in Parkinson's disease patients

Various physiotherapeutic methods and approaches play a significant role in the treatment of patients with Parkinson’s disease, including the use of enriched environments. Virtual reality (VR) as a type of enriched environment has the potential to create multiple sensory experiences and feedback, influencing various aspects of the patient’s information processing and response. The suitability for home use and the considerable impact on motivation highlight its advantages over alternative approaches. The objective of this review is to investigate the impact of VR-based exercise on balance outcomes among individuals with Parkinson’s disease. The inclusion criteria consisted of randomized controlled trials (RCTs) that examined the effects of exercise in a VR environment on individuals’ static and dynamic balance outcomes. In order to gather relevant studies, we conducted a comprehensive search across three databases. From a dataset of 625 records, we conducted a comprehensive full-text screening based on specific inclusion and exclusion criteria. This process resulted in the inclusion of 14 RCTs in our review. The emerging evidence regarding exercising in a VR environment does not definitively prove its superiority over standard exercise routines. However, studies have demonstrated that both the experimental and control groups showed comparable improvements in enhancing static and dynamic balance among individuals with Parkinson’s disease. The comparable improvements in balance observed between the experimental and control groups signify the potential effectiveness of VR-based exercises. This underscores the encouragement for further development in this technology, particularly focusing on fully immersive VR environments, which may yield superior effects in enhancing balance among individuals with Parkinson’s disease.

A virtual reality environment for supporting mental wellbeing of students on remote clinical placement: A multi-methods evaluation

BackgroundNursing and Allied Health Profession (NAHP) students undertake clinical placements as part of their pre-registration training. The remote nature of some placement sites, shiftwork and the emotionally challenging nature of the workload has led to mental wellbeing issues in many students. AimThis project aimed to evaluate a novel 3D immersive virtual reality environment that supports mental wellbeing for NAHP students on clinical placement. It comprises a calming 3D tropical beach environment where students and tutors can meet for reflection and mutual support. DesignA multi-methods design gathered quantitative impact data with validated measurement tools and qualitative output related to the lived experience of students. Settings and participantsAll 600 pre-registration NAHP students within the institution undertaking clinical placements were invited to participate, irrespective of mental wellbeing status. Students were randomly assigned to either a VR or Conventional cohort; all participants received the control support mechanism in a subsequent placement. MethodsAll participants completed an initial demographic and Readiness for Therapy survey followed by weekly Beck Anxiety and Depression Inventories during placement. All participants were invited to a semi-structured interview. ResultsOverall, 32 participants engaged with the application; although the VR cohort demonstrated improved scores on both Beck inventories, these were not statistically significant. This is probably due to the low response rate for the control cohort. A total of 15 interviews were conducted and several themes emerged from the data in relation.to both experiential outcomes (escapism, anonymity and sense of community) and instrumental outcomes (calming, mindfulness and combatting loneliness). ConclusionsUser feedback indicates that a VR environment can provide a calming escape from the pressures and anxiety arising from clinical placement for healthcare students. The relaxing beach environment facilitated mindfulness meditation and the additional opportunities for pseudo-anonymous interactions with peers and tutors were wellreceived by students.

Preparing teachers for the future: Microteaching in the immersive VR environment

Abstract The present study aims to explore how pre- and in-service language teachers incorporate the cutting-edge technology of immersive virtual reality (iVR) into their teaching practice. Specifically, the study examined how their different knowledge levels and teaching experiences influenced their integration of technology by analyzing their performance-based tasks in microteaching in an iVR environment. This particular technology was selected for the study because it was expected to bring multiple pedagogical benefits to future foreign language learning classrooms, such as contextualized learning, increased learner motivation and interest, and enhanced interaction and communicative skill training. The study employed in-depth qualitative analysis. Data (lesson plans, screen recordings of microteaching episodes, and reflection papers) were collected from one preservice teacher training course and one in-service teacher training course at a Korean university. The study found a large gap between pre- and in-service teacher performance and identified the sources of the differences based on qualitative data analysis. The results showed that not only teachers’ technological knowledge but also their pedagogical knowledge of the use of technology and confidence in teaching affected technology integration. As technology integration has become more important in language education, the current study provided insight into how to better prepare teachers for future learners.

Virtual reality platform for teacher training on classroom climate management: evaluating user acceptance

Enhancing the educational experience through Immersive Virtual Reality (IVR) is a promising avenue, elevating the authenticity and responsiveness of simulations. Particularly in educational settings, IVR holds the potential to augment accessibility and engagement in learning. However, one pivotal aspect lies in assessing the learners' acceptance of such environments to ensure optimal and effective utilization of these technologies. This paper delves into the Didascalia Virtual-ClassRoom usability testing —an immersive IVR environment tailored for pre-service secondary school teachers. The platform transports users into a simulated classroom, where they are invited to play the role of a teacher. During the simulation, three scenarios are recreated, reproducing disruptive behaviours commonly faced in real classrooms. 84 participants (28 teachers and 56 pre-service teachers) engaged in decision-making to manage the classroom climate influenced by the simulated situations. To collect data, we used a questionnaire based on the Technology Acceptance Model (TAM) to assess and gauge users' inclinations and attitudes towards embracing the technology in question. To gain deeper insights into the user experience, participants were further invited to participate in semi-structured interviews, offering reflections and suggestions for potential enhancements. The evaluation process encompassed the perceived usefulness of the Didascalia Virtual-ClassRoom, shedding light on factors that could either facilitate or impede the adoption of this platform to enhance classroom management competence. The participants' perspectives serve as a valuable foundation for refining the tool's functionality, and their feedback fuels recommendations for its seamless integration into initial teacher training programs.

Immersive Virtual Reality Serious Game Studies in Education: A Brief Review

Currently, immersive virtual reality environments(IVR) are providing interactive learning experiences in various fields including “education”. Especially after covid-19 pandemic, the popularity of the online learning environment has increased, and the IVR would be a possible alternative to other mediums in online learning. The previous reviews usually contain related literature before the pandemic. There is a need to better understand the current body of knowledge on IVR. The specific application contexts, research designs, and possible results of the current works needed to be clarified. To answer this question a literature review was conducted between 2013-2023 years on the “Web of Science” database for “IVR” and “serious games” keywords in the field of “education”. Then, 18 articles are reviewed and the main themes are provided. As a result, the majority of the articles were found as “applied studies” and the main research intent was to determine “cognitive effects” including knowledge improvement. Depending on literature analyses qualitative and quantitative study results can be listed as; “IVR in the form of serious game” in general was found and perceived as an effective medium for learning in various fields of education. In addition, some study results about instructional design decisions of IVR applications and the effectiveness of IVR comparing other types of mediums are provided. The study recommends taking into account educational theories behind the design of IVR and related multimedia design principles.

Virtual Access to STEM Careers: In the Field Experiments.

The Virtual Access to STEM Careers (VASC) project is an intertwined classroom and virtual reality (VR) curricular program for third through fourth graders. Elementary school students learn about and take on the roles and responsibilities of STEM occupations through authentic, problem-based tasks with physical kits and immersive VR environments. This article reports on a round of curriculum and virtual environment development and in-classroom experimentation that was guided by preliminary results gathered from our initial VASC prototyping and testing. This specific iteration focuses on curriculum for learning about sea turtles and tasks regularly completed by park rangers and marine biologists who work with these creatures and a new backend data collection component to analyze participant behavior. Our results showed that educators were able to setup and integrate VASC into their classrooms with relative ease. Elementary school students were able to learn how to interface with our system quickly and enjoyed being in the environment, making a positive link to STEM education.

A method for synchronized use of EEG and eye tracking in fully immersive VR.

This study explores the synchronization of multimodal physiological data streams, in particular, the integration of electroencephalography (EEG) with a virtual reality (VR) headset featuring eye-tracking capabilities. A potential use case for the synchronized data streams is demonstrated by implementing a hybrid steady-state visually evoked potential (SSVEP) based brain-computer interface (BCI) speller within a fully immersive VR environment. The hardware latency analysis reveals an average offset of 36 ms between EEG and eye-tracking data streams and a mean jitter of 5.76 ms. The study further presents a proof of concept brain-computer interface (BCI) speller in VR, showcasing its potential for real-world applications. The findings highlight the feasibility of combining commercial EEG and VR technologies for neuroscientific research and open new avenues for studying brain activity in ecologically valid VR environments. Future research could focus on refining the synchronization methods and exploring applications in various contexts, such as learning and social interactions.

The effects of immersion in virtual reality environment on oral English learning for Chinese university students

The effects of immersion in virtual reality environment on oral English learning for Chinese university students

Exploring students’ immersive VR experiences as resources for collaborative meaning making and learning

By adopting a situated and interactional approach, we explore students’ immersive VR experiences as resources for collaborative meaning making and learning. We draw on data from a project in which teachers and researchers co-developed a learning design for upper secondary students involving immersive VR technology. In this learning design, students viewed a cinematic VR film where they encountered different people telling personal stories about exclusion and discrimination, followed by reflective group dialogues with their teacher about their experiences in this environment. Through a detailed interaction analysis of these dialogues, we identify four dimensions that characterize students’ meaning making: (1) the feeling of taking part in conversations, (2) attending to bodily expressions of others, (3) students’ own bodily responses, (4) teacher guidance. We discuss how the findings from our analysis contribute to the field of CSCL, and which also have implications for instructional work that includes the use of immersive VR environments.

A virtual reality data visualization tool for dimensionality reduction methods

In this paper, we present a virtual reality interactive tool for generating and manipulating visualizations for high-dimensional data in a natural and intuitive stereoscopic way. Our tool offers support for a diverse range of dimensionality reduction (DR) algorithms, enabling the transformation of complex data into insightful 2D or 3D representations within an immersive VR environment. The tool also allows users to include annotations with a virtual pen using hand tracking, to assign class labels to the data observations, and to perform simultaneous visualization with other users within the 3D environment to facilitate collaboration.

Three Virtual Reality Environments for the Assessment of Executive Functioning Using Performance Scores and Kinematics: An Embodied and Ecological Approach to Cognition.

Virtual reality (VR) is an attractive technology for cognitive assessment, as it provides a more embodied experience compared with typical test situations, such as those using paper and pencil. In addition, VR can immerse individuals in complex situations similar to real-life ones, thereby improving the ecological validity (i.e., face validity) of the assessment. VR also offers improved scoring of tests as it facilitates the tracking of kinematic information and the temporal tracking of activities. This study assesses the correlation between scores on executive function assessments using standard neuropsychological tasks in paper-and-pencil format, on a tablet, and in three immersive VR environments, each designed to involve specific aspects of executive function. This study also aims to assess the correlation between these performance scores and a set of kinematic measures (speed, duration, and distance traveled by the hand) collected in VR. The outcomes, including performance scores and kinematic measures, correlate both with traditional assessment methods (such as paper and pencil, and computerized 2D tests) and with each other, suggesting their potential usefulness in clinical and research contexts. The discussion focuses on the advantages of embodied, situated, and spatialized tests for cognitive assessment and the benefits of kinematic tracking in VR tests for the quality of this assessment.

Manipulating objects during learning shrinks the global scale of spatial representations in memory: a virtual reality study

Goal-directed approaches to perception usually consider that distance perception is shaped by the body and its potential for interaction. Although this phenomenon has been extensively investigated in the field of perception, little is known about the effect of motor interactions on memory, and how they shape the global representation of large-scale spaces. To investigate this question, we designed an immersive virtual reality environment in which participants had to learn the positions of several items. Half of the participants had to physically (but virtually) grab the items with their hand and drop them at specified locations (active condition). The other half of the participants were simply shown the items which appeared at the specified position without interacting with them (passive condition). Half of the items used during learning were images of manipulable objects, and the other half were non manipulable objects. Participants were subsequently asked to draw a map of the virtual environment from memory, and to position all the items in it. Results show that active participants recalled the global shape of the spatial layout less precisely, and made more absolute distance errors than passive participants. Moreover, global scaling compression bias was higher for active participants than for passive participants. Interestingly, manipulable items showed a greater compression bias compared to non-manipulable items, yet they had no effect on correlation scores and absolute non-directional distance errors. These results are discussed according to grounded approaches of spatial cognition, emphasizing motor simulation as a possible mechanism for position retrieval from memory.