Immersive 3D Research Articles

Towards Mineralogy 4.0? Atlas of 3D Rocks and Minerals: Digitally Archiving Interactive and Immersive 3D Data for Rocks and Minerals

Mineralogy 4.0 can play a significant role in the future of geological research, education, and exploration by providing a more comprehensive and interactive understanding of rocks and minerals. This paper explores the application of digital photogrammetry and augmented reality (AR) technologies as part of Mineralogy 4.0. An atlas of 3D rocks and minerals with 915 high-quality models was created to showcase the potential of photogrammetry in the mineral sciences. The repository contains a wide range of sample types, featuring transparency, metallic luster, fluorescence, or millimetric-scale crystals. The three-dimensional rocks and minerals can also be accessed on-the-go through a mobile application that was developed for Android devices. Additionally, web applications have been developed with specific three-dimensional collections as well as three-dimensional storytelling. AR technology was also integrated into the 3D repository, allowing users to superimpose virtual 3D models of rocks and minerals onto real-world surfaces through their device’s camera. Also, a digital solution with 3D holograms of rocks and minerals was effectively implemented to provide an interactive and immersive experience. The 3D datasets of rocks and minerals can play a significant role in the geoscience community’s research, developing not only in-depth knowledge of specimens but also opening new frontiers in mineral sciences, leading towards a more advanced era of mineralogy.

Image modeling algorithm for environment design based on augmented and virtual reality technologies

Abstract With the continuous development of computer vision technology, the image modeling of environmental design has also received more and more attention. This study proposed an image modeling algorithm for environment design based on augmented reality (AR) and virtual reality (VR) technologies. On the system architecture, the environment design image modeling algorithm based on AR and VR technologies has provided users with an immersive 3D simulation space environment. It also has provided users with a good interactive interface through AR technology, enabling users to interact with computers in a realistic virtual three-dimensional environment and interactive operation. This study verified the algorithm from three aspects: time, authenticity, and interactivity. When facing object A, the modeling time values of the traditional environmental design image modeling method and the environmental design image modeling method proposed in this study were 12.14 and 8.72 s, respectively. The modeling authenticity reached 87.51 and 98.57%, respectively, and the interactivity reached 79.14 and 96.45%, respectively. Some columns of data have proved that the environmental design image modeling method proposed in this study was superior to the traditional environmental design image modeling method. In addition, this study also verified the system model from three aspects: stability, running smoothness, and performance loss. When the number of image modeling in the environment design was 2, the stability, running smoothness, and performance loss of the system were 99.37, 99.25 and 0.75%, respectively. The experimental data again proved that the algorithm system model proposed in this study was feasible and worthy of further application.

Metaverse-based cardiac magnetic resonance simulation to overcome claustrophobia (META-MRI): a feasibility trial

Abstract Background Cardiac Magnetic Resonance Imaging (CMR) is crucial for diagnosing and stratifying cardiac disease, but up to 10% of patients face challenges due to claustrophobia. Conventional methods for overcoming claustrophobia have limited success. The metaverse, an immersive 3D environment using virtual and augmented reality, is under investigation for healthcare applications. A metaverse-based VR CMR simulation program was developed to assist patients with claustrophobia, but its clinical effectiveness is unexplored. Methods The META-MRI study involved 450 patients screened, with 32 enrolled. They were randomly assigned to intervention (Metaverse-based simulation + standard care) or control (standard care alone). The Metaverse program used a VR headset within the Aimedis metaverse platform. Training sessions occurred before scheduled CMR exams, with anxiolytic treatment when necessary. Primary endpoints included successful CMR completion, anxiety assessment using the State-Trait Anxiety Inventory (STAI), and usability through a questionnaire. Results Patients in the intervention group (n=16) completed Metaverse training successfully. During simulations, anxiety decreased (STAI scores: 73 ± 8 to 51 ± 11, p=0.02). Intervention group completion rate was significantly higher (81%) than the control group (37%) during real CMR exams (p=0.01). Usability scores indicated moderate satisfaction. Discussion The metaverse-based program significantly improved CMR completion rates and reduced anxiety levels in patients with severe claustrophobia compared to standard care alone. Traditional methods for claustrophobia have limitations, making the metaverse a promising tool. Despite promising results, further research is necessary, considering identified usability issues and economic implications. Conclusion This feasibility trial demonstrates the effectiveness of a metaverse-based simulation program in enhancing compliance with CMR exams for patients with severe claustrophobia. The strategy, combined with standard care, led to reduced anxiety and increased CMR completion. Further investigation with larger cohorts is required to validate these findings, addressing usability concerns and assessing the cost-effectiveness of incorporating the metaverse into routine clinical practice.Clinical and demographic characteristics

Design and development of immersive 3D virtual simulation experiment teaching platform for internet of things

Design and development of immersive 3D virtual simulation experiment teaching platform for internet of things

Virtual Reality-Assisted Informed Consent for Anesthesia: A Prospective and Randomized Proof-of-Concept Study.

Background/Objectives: Informed consent for anesthesia poses both legal challenges and problems of understandable communication. Fulfilling all the requirements through anesthesiologists directly interacting with patients is a time- and staff-consuming strategy. Given today's smart technologies, notably including virtual reality (VR), we explored in a prospective randomized study whether 'VR-assisted informed consent' could improve this situation. Methods: Fifty patients scheduled for orthopedic surgery were randomized. In the control group, informed consent was obtained via patient-specialist dialogs only. The patients in the study group, wearing a head-mounted display, watched an 8 min immersive 3D movie with the standard explanations of general anesthesia, followed by a patient-specialist dialog to address open questions. The time spent on the dialogs in both groups was evaluated as the primary outcome variable. As secondary variables, we analyzed both a three-item Likert scale on patient satisfaction with the VR experience and cost differences between both groups. Results: Patient-specialist dialogs were carried on for median (IQR) durations of 93 (20-182) seconds in the study group versus 665 (261-829) seconds in the control group (p < 0.001). All the patients exposed to VR rated this experience as favorable (87.5%) or neutral (12.5%). Based on anesthesiologists' incomes in the US and UK, our approach would reduce the staff expenditure for each patient-specialist dialog by median amounts of USD ≈40 or ≈11, respectively (2 × p < 0.001). Conclusions: 'VR-assisted informed consent' for anesthesia is well accepted by patients and reduces the time requirements for patient-specialist dialogs, thus pointing out a potential avenue towards increasing the work-time efficiency of anesthesiologists.

Modelling the effects of perceived system quality and personal innovativeness on the intention to use metaverse: a structural equation modelling approach

The metaverse, an interactive and immersive 3D virtual environment, has recently become popular and is widely used in several fields, including education. However, the successful use of metaverse relies on the extent to which users intend to adopt and use it. Close examination of this critical issue reveals a lack of research that examines the effects of certain factors on users’ intentions toward using metaverses. Thus, this study extends the technology acceptance model by integrating two constructs—perceived system quality and students’ personal innovativeness. Using a survey to collect data, 164 responses were received from students at the University of Ha’il in Saudi Arabia. Two steps in structural equation modelling (SEM) using the AMOS software were applied to analyse the data and test the research hypotheses. The results revealed that perceived system quality had a significant effect on students’ intentions to use metaverses through perceived ease of use. Furthermore, personal innovativeness had a significant effect on students’ intentions through the perceived usefulness of the metaverse. In addition, perceived usefulness affected students’ intentions to use a metaverse. Surprisingly, perceived ease of use had an insignificant effect on students’ intentions to use the metaverse. Although the proposed model and its findings contribute to the technology acceptance model (TAM) literature, the study’s practical value is significant because it can help educational policymakers and authorities to understand the effect of each factor and plan future strategies. Additionally, the findings of this study can assist practitioners, designers, and developers in designing and promoting the utilisation of metaverses.

3D audio recording of complex underwater sound fields

In this study, we developed a method to capture underwater sound fields and reproduce them for human listeners, providing an immersive 3D auditory experience. We recorded the sound field with an array of four hydrophones arranged in a tetrahedral structure. The side length of the tetrahedron (around 70 cm) was a compromise between capturing accurately the low frequency pressure gradients and spatial aliasing effects at higher frequencies. Subsequently, we transformed the signals into the domain of spherical harmonics facilitating a representation as first-order Ambisonics that is independent of the recording. For playback on devices such as headphones or loudspeaker systems, we used established decoding routines that enhance the immersive audio experience with techniques such as parametric decoding and diffuse sound field equalization. During an expedition to northern Norway, we recorded the underwater soundscape in deep fjords with vocalizing killer whales and singing humpback whales. The resulting binaural renderings for headphones of these sound scenes are astonishing and sparked interest among scientist, artists and general public. This research also helps to enhance our understanding and perception of these intelligent marine mammals and their unique acoustic underwater world, ultimately aiding in better protecting them from noise pollution.

Full-Body Pose Estimation of Humanoid Robots Using Head-Worn Cameras for Digital Human-Augmented Robotic Telepresence

We envision a telepresence system that enhances remote work by facilitating both physical and immersive visual interactions between individuals. However, during robot teleoperation, communication often lacks realism, as users see the robot’s body rather than the remote individual. To address this, we propose a method for overlaying a digital human model onto a humanoid robot using XR visualization, enabling an immersive 3D telepresence experience. Our approach employs a learning-based method to estimate the 2D poses of the humanoid robot from head-worn stereo views, leveraging a newly collected dataset of full-body poses for humanoid robots. The stereo 2D poses and sparse inertial measurements from the remote operator are optimized to compute 3D poses over time. The digital human is localized from the perspective of a continuously moving observer, utilizing the estimated 3D pose of the humanoid robot. Our moving camera-based pose estimation method does not rely on any markers or external knowledge of the robot’s status, effectively overcoming challenges such as marker occlusion, calibration issues, and dependencies on headset tracking errors. We demonstrate the system in a remote physical training scenario, achieving real-time performance at 40 fps, which enables simultaneous immersive and physical interactions. Experimental results show that our learning-based 3D pose estimation method, which operates without prior knowledge of the robot, significantly outperforms alternative approaches requiring the robot’s global pose, particularly during rapid headset movements, achieving markerless digital human augmentation from head-worn views.

Ten Years of Operating a Center with Large-Scale Virtual-Reality Installations: Developments and Learnings

Abstract Virtual reality (VR) can be found in many fields including art history, education, research, and industry 4.0. Next to lightweight head-mounted displays there are also VR installations, such as immersive 3D screens, large-scale displays, and CAVE systems, that are used in research, scientific visualization, and also the automotive industry. These systems offer high visual quality and collaborative VR experiences for researchers and have long been used in research. We present learnings and insights from ten years of operating and maintaining a visualization center with large-scale immersive displays and installations. Our report aims to answer questions on the benefits and challenges of such a location-based VR center. We broke down the ten years into three phases and discuss the installations themselves, software, and other various developments of the center over time, illustrated with exemplary use cases. Finally, we cover our experiences at the beginning of phase four with the installation of a (novel) LED CAVE and the future (forward-looking) developments we expect for location-based VR centers.

Enhancing educational experience through establishing a VR database in craniosynostosis: report from a single institute and systematic literature review.

Craniosynostosis is a type of skull deformity caused by premature ossification of cranial sutures in children. Given its variability and anatomical complexity, three-dimensional visualization is crucial for effective teaching and understanding. We developed a VR database with 3D models to depict these deformities and evaluated its impact on teaching efficiency, motivation, and memorability. We included all craniosynostosis cases with preoperative CT imaging treated at our institution from 2012 to 2022. Preoperative CT scans were imported into SpectoVR using a transfer function to visualize bony structures. Measurements, sub-segmentation, and anatomical teaching were performed in a fully immersive 3D VR experience using a headset. Teaching sessions were conducted in group settings where students and medical personnel explored and discussed the 3D models together, guided by a host. Participants' experiences were evaluated with a questionnaire assessing understanding, memorization, and motivation on a scale from 1 (poor) to 5 (outstanding). The questionnaire showed high satisfaction scores (mean 4.49 ± 0.25). Participants (n = 17) found the VR models comprehensible and navigable (mean 4.47 ± 0.62), with intuitive operation (mean 4.35 ± 0.79). Understanding pathology (mean 4.29 ± 0.77) and surgical procedures (mean 4.63 ± 0.5) was very satisfactory. The models improved anatomical visualization (mean 4.71 ± 0.47) and teaching effectiveness (mean 4.76 ± 0.56), with participants reporting enhanced comprehension and memorization, leading to an efficient learning process. Establishing a 3D VR database for teaching craniosynostosis shows advantages in understanding and memorization and increases motivation for the study process, thereby allowing for more efficient learning. Future applications in patient consent and teaching in other medical areas should be explored.

User preferences for 2D markings of public buildings in immersive 3D geovisualizations

User preferences for 2D markings of public buildings in immersive 3D geovisualizations

XR-based interactive visualization platform for real-time exploring dynamic earth science data

The transition from 2D planar displays to immersive holographic 3D environments has brought advancements in visualization technology. However, there remains a lack of effective interactive visualization tools for complex multi-dimensional structured or unstructured datasets in immersive space. To address this gap, we have developed MetIVA, a state-of-the-art multiscale interactive data visualization platform that leverages the Extended Reality (XR) and cloud rendering technology for immersive data exploration. In this paper, we firstly outline the historical development of scientific visualization and the recent shift towards 3D and higher-dimensional visualization, and then basically introduce the conceptual framework and platform structure of MetIVA, and finally present the evaluation results from recruited potential users. The results confirm that MetIVA is a powerful tool to accelerate data exploration and decision-making processes. Its interactive and intuitive features, along with ongoing optimization efforts, make it a valuable tool for researchers and practitioners in the field of Earth science.

Heritage elements and Artificial Intelligence as storytelling tools for virtual retail environments

In the rapidly evolving landscape of technology, the metaverse has emerged as a groundbreaking platform for redefining fashion presentation, experience, and conceptualization. This paper investigates the interplay between fashion and Virtual Reality (VR) through specific use cases of virtual showrooms. Each case highlights the metaverse's potential in fashion, from reproposing historical elements to expressing contemporary trends, showcasing technological innovations, like Artificial Intelligence (AI), and user engagement strategies. This analysis shows how such commercial spaces can be exploratory platforms for narrating a brand's history and heritage elements, revealing VR's potential to enhance user engagement with immersive 3D environments and dynamic narratives. The paper further investigates advancements in integrating heritage with e-commerce elements and using heritage and AI as storytelling tools, concentrating on four exemplary projects. This integration suggests a future of increasingly immersive, personalized, and interactive experiences, marking a new chapter in how fashion is conceived, presented, and experienced.

Force-Directed Immersive 3D Network Visualization

Network visualization, in mathematics often referred to as graph visualization, has evolved significantly over time, offering various methods to effectively represent complex data structures. New methods and devices advance the possibilities of visualization both from the point of view of the quality of displayed information and of the possibilities of visualizing a larger amount of data. Immersive visualization includes the user directly in presented visual representation but requires a native 3D environment for direct interaction with visualized information. This article describes an approach to creating a force-directed immersive 3D network visualization algorithm available for application in immersive environments, such as a cave automatic virtual environment or virtual reality. The algorithm aims to address the challenge of creating visually appealing and easily interpretable visualizations by utilizing 3D space and the Unity engine. The results show successfully visualized data and developed interactive visualization methods, overcoming limitations of basic force-directed implementations. The main contribution of the presented research is the force-directed algorithm with springs and controlled placement as an immersive visualization technique that combines the use of springs and attractive forces to stabilize a network in a 3D environment.

Uncertain Future Dwelling: Emergent Interiors of the Metaverse

Contemporarily, a flood of digital interior architectural imagery has emerged of spaces developed for the Metaverse, a forthcoming immersive 3D virtual world. These spaces are not bound by the conventions of architectural practice nor the demands of the physical world, providing an opportunity for design exploration and innovation in the future of interiors and positing challenges to core architectural concepts that have accompanied traditional practice. This research offers a visual analysis of aesthetic trends and new typologies present in the interior architectural spaces designed for the Metaverse. The analysis features a curated and collaged collection of works from ten creators of Metaverse spaces, categorised to examine the impact of digital architectural spaces that increasingly detach from the needs of physical dwellings. The research reveals commanding visual trends in Metaverse interior imagery that challenge traditional notions of interiority and dwelling and finds aesthetic signifiers of belonging in spaces that Augé (1995/2009) would neatly classified as an empirical 'non-place.' Positioned as a form of heterotopia in a realm where architecture is being designed for the purely visual, we posit that the less recognisable these spaces become, the more potential they hold for innovation in both the Metaverse and in dialogue with real-world interior architecture.

Virtual Inspection System for Pumping Stations with Multimodal Feedback.

Pumping stations have undergone significant modernization and digitalization in recent decades. However, traditional virtual inspections often prioritize the visual experience and fail to effectively represent the haptic physical properties of devices during inspections, resulting in poor immersion and interactivity. This paper presents a novel virtual inspection system for pumping stations, incorporating virtual reality interaction and haptic force feedback technology to enhance immersion and realism. The system leverages a 3D model, crafted in 3Ds Max, to provide immersive visualizations. Multimodal feedback is achieved through a combination of haptic force feedback provided by a haptic device and visual information delivered by a VR headset. The system's data platform integrates with external databases using Unity3D to display relevant information. The system provides immersive 3D visualizations and realistic force feedback during simulated inspections. We compared this system to a traditional virtual inspection method that demonstrated statistically significant improvements in task completion rates and a reduction in failure rates when using the multimodal feedback approach. This innovative approach holds the potential to enhance inspection safety, efficiency, and effectiveness in the pumping station industry.

The Conceptual Model Mediated by IVR and 3DP as a First Architectural Idea Generator

The first ideas generation of the architectural project has traditionally been carried out through sketches. Even though in recent decades models have also gained importance, their use is still restricted due to the time required to make them and their difficulty of realization when it comes to complex forms. This research argues that the use of concept models mediated by two disruptive technologies such as Immersive Virtual Reality and 3D Printing can foster the cognitive process of the ideation and configuration of the first architectural ideas. To demonstrate the hypothesis, a pilot study was conducted with 32 architecture students; 17 students created models manually and 15 used IVR and 3DP. Two observation sheets were used to collect information. The results show that in the process of generating ideas, the group that used the two technologies developed three of the four characteristics of the cognitive process better, being an undecided and slow process, then fast and continuous, and finally perceptual for reinterpretation. Also, both technologies complemented each other and favored the development of an intense, tactile, and phenomenological experience. Finally, regarding the model as a product, a better result was found in 3D-printed models in terms of two of the three characteristics, three-dimensionality complexity and materiality. It is concluded that these two technologies ideally complement each other as mediating tools for three-dimensional architectural thinking, making it possible to use the conceptual models as the first generators of the architectural idea.

Human brain dynamics dissociate early perceptual and late motor-related stages of affordance processing.

Affordances, the opportunities for action offered by the environment to an agent, are vital for meaningful behaviour and exist in every interaction with the environment. There is an ongoing debate in the field about whether the perception of affordances is an automated process. Some studies suggest that affordance perception is an automated process that is independent from the visual context and bodily interaction with the environment, whereas others argue that it is modulated by the visual and motor context in which affordances are perceived. The present paper aims to resolve this debate by examining affordance automaticity from the perspective of sensorimotor time windows. To investigate the impact of different forms of bodily interactions with an environment, that is, the movement context (physical vs. joystick movement), we replicated aprevious study on affordance perception in which participants actively moved through differently wide doors in an immersive 3D virtual environment. In the present study, wedisplayed the same environmenton a 2D screen with participants moving through doors of different widths using the keys on a standard keyboard. We compared components of the event-related potential (ERP) from the continuously recorded electroencephalogram (EEG) that were previously reported to be related to affordance perception of architectural transitions (passable and impassable doors). Comparing early sensory and later motor-related ERPs, our study replicated ERPs reflecting early affordance perception but found differences in later motor-related components. These results indicate a shift from automated perception of affordances during early sensorimotor time windows to movement context dependence of affordance perception at later stages, suggesting that affordance perception is a dynamic and flexible process that changes over sensorimotor stages.

Can viewing a 3D movie improve visual function in children with a history of amblyopia and neurotypical children?: A pilot study.

The aim of this pilot study was to determine whether viewing an immersive 3D movie with large disparities in a cinema resulted in improved visual acuity (VA), stereoscopic depth perception (ST), and improved eye alignment in residual amblyopic children and children without amblyopia. A total of 24 children aged between 5 and 12 years with a history of anisometropic and/or strabismic amblyopia, that had been previously treated and who currently have residual amblyopia (N = 14), and in children with typical development without amblyopia (N = 10) viewed the movie in 3D Sing 2 in a cinema for 110 minutes. Visual acuity, stereoacuity and ocular deviation were assessed before viewing the movie, and three months later. Stereoacuity and ocular deviation were also measured immediately after viewing the movie. We observed an improvement in visual acuity in the non-dominant (amblyopic) eye 3 months after viewing the movie in the amblyopic group (P<0.001). Stereopsis improved immediately after viewing the movie (P = 0.02), and after 3 months by ≈ 40% (P = 0.01). Moreover, improvements in stereopsis were also observed in children without amblyopia (P = 0.04). No significant changes in ocular deviation were observed in either group. These pilot results suggest that brief exposure to large disparities by viewing a 3D movie in a cinema can help to improve stereopsis and visual acuity in children aged 5‒12 years with previously treated amblyopia, and provide a rationale for a randomized clinical trial.

Urban Green Spaces and Mental Well-Being: A Systematic Review of Studies Comparing Virtual Reality versus Real Nature

Increasingly, urban planners are adopting virtual reality (VR) in designing urban green spaces (UGS) to visualize landscape designs in immersive 3D. However, the psychological effect of green spaces from the experience in VR may differ from the actual experience in the real world. In this paper, we systematically reviewed studies in the literature that conducted experiments to investigate the psychological benefits of nature in both VR and the real world to study nature in VR anchored to nature in the real world. We separated these studies based on the type of VR setup used, specifically, 360-degree video or 3D virtual environment, and established a framework of commonly used standard questionnaires used to measure the perceived mental states. The most common questionnaires include Positive and Negative Affect Schedule (PANAS), Perceived Restorativeness Scale (PRS), and Restoration Outcome Scale (ROS). Although the results from studies that used 360-degree video were less clear, results from studies that used 3D virtual environments provided evidence that virtual nature is comparable to real-world nature and thus showed promise that UGS designs in VR can transfer into real-world designs to yield similar physiological effects.