2D Screen Research Articles

Demonstration of Immersive Technologies for Geospatial Learning

Abstract. Immersive technologies are becoming a powerful tool for educators across multiple disciplines including geospatial science. They offer new ways to engage and educate geospatial students, removing barriers that may exist in traditional teaching methods. Especially because the limitations of 2D screens are often exceeded by the complexity of modern data sets. Examples of challenges that educators often encounter are explaining theoretical concepts in the classroom, providing alternate scenarios, preparing students for physical labs, and limited / restricted access to physical sites. Immersive technologies can be a great resource to support on-site lectures and enhance remote learning, a necessity in today's educational panorama. Immersive technologies typically include virtual, augmented, and mixed reality. Each method offers different pedagogical advantages and poses different challenges. Before any implementation, these advantages and challenges must be examined and understood to maximize the benefit of immersive methods to the students and mitigate potential drawbacks that could hinder learning outcomes. This paper provides a review of the different immersive methods, presenting examples of their application in geospatial education with lessons learned and recommendations for future work. The case examples include a variety of different instruments and tasks such as the simulation of GNSS, differential leveling, total station operations, and airborne LiDAR data collection.

Assessment of Interrater Reliability and Accuracy of Cerebral Aneurysm Morphometry Using 3D Virtual Reality, 2D Digital Subtraction Angiography, and 3D Reconstruction: A Randomized Comparative Study.

Detailed morphometric analysis of an aneurysm and the related vascular bifurcation are critical factors when determining rupture risk and planning treatment for unruptured intracranial aneurysms (UIAs). The standard visualization of digital subtraction angiography (DSA) and its 3D reconstruction on a 2D monitor provide precise measurements but are subject to variability based on the rater. Visualization using virtual (VR) and augmented reality platforms can overcome those limitations. It is, however, unclear whether accurate measurements of the aneurysm and adjacent arterial branches can be obtained on VR models. This study aimed to assess interrater reliability and compare measurements between 3D VR, standard 2D DSA, and 3D DSA reconstructions, evaluating the reliability and accuracy of 3D VR as a measurement tool. A pool of five neurosurgeons performed three individual analyses on each of the ten UIA cases, measuring them in completely immersed 3D VR and the standard on-screen format (2D DSA and 3D reconstruction). This resulted in three independent measurements per modality for each case. Interrater reliability of measurements and morphology characterization, comparative differences, measurement duration, and VR user experience were assessed. Interrater reliability for 3D VR measurements was significantly higher than for 3D DSA measurements (3D VR mean intraclass correlation coefficient [ICC]: 0.69 ± 0.22 vs. 3D DSA mean ICC: 0.36 ± 0.37, p = 0.042). No significant difference was observed between 3D VR and 2D DSA (3D VR mean ICC: 0.69 ± 0.22 vs. 2D DSA mean ICC: 0.43 ± 0.31, p = 0.12). A linear mixed-effects model showed no effect of 3D VR and 3D DSA (95% CI = -0.26-0.28, p = 0.96) or 3D VR and 2D DSA (95% CI = -0.02-0.53, p = 0.066) on absolute measurements of the aneurysm in the anteroposterior, mediolateral, and craniocaudal dimensions. 3D VR technology allows for reproducible, accurate, and reliable measurements comparable to measurements performed on a 2D screen. It may also potentially improve precision for measurements of non-planar aneurysm dimensions.

Revisiting the Heider and Simmel experiment for social meaning attribution in virtual reality

In their seminal experiment in 1944, Heider and Simmel revealed that humans have a pronounced tendency to impose narrative meaning even in the presence of simple animations of geometric shapes. Despite the shapes having no discernible features or emotions, participants attributed strong social context, meaningful interactions, and even emotions to them. This experiment, run on traditional 2D displays has since had a significant impact on fields ranging from psychology to narrative storytelling. Virtual Reality (VR), on the other hand, offers a significantly new viewing paradigm, a fundamentally different type of experience with the potential to enhance presence, engagement and immersion. In this work, we explore and analyze to what extent the findings of the original experiment by Heider and Simmel carry over into a VR setting. We replicate such experiment in both traditional 2D displays and with a head mounted display (HMD) in VR, and use both subjective (questionnaire-based) and objective (eye-tracking) metrics to record the observers’ visual behavior. We perform a thorough analysis of this data, and propose novel metrics for assessing the observers’ visual behavior. Our questionnaire-based results suggest that participants who viewed the animation through a VR headset developed stronger emotional connections with the geometric shapes than those who viewed it on a traditional 2D screen. Additionally, the analysis of our eye-tracking data indicates that participants who watched the animation in VR exhibited fewer shifts in gaze, suggesting greater engagement with the action. However, we did not find evidence of differences in how subjects perceived the roles of the shapes, with both groups interpreting the animation’s plot at the same level of accuracy. Our findings may have important implications for future psychological research using VR, especially regarding our understanding of social cognition and emotions.

Human brain dynamics dissociate early perceptual and late motor-related stages of affordance processing.

Affordances, the opportunities for action offered by the environment to an agent, are vital for meaningful behaviour and exist in every interaction with the environment. There is an ongoing debate in the field about whether the perception of affordances is an automated process. Some studies suggest that affordance perception is an automated process that is independent from the visual context and bodily interaction with the environment, whereas others argue that it is modulated by the visual and motor context in which affordances are perceived. The present paper aims to resolve this debate by examining affordance automaticity from the perspective of sensorimotor time windows. To investigate the impact of different forms of bodily interactions with an environment, that is, the movement context (physical vs. joystick movement), we replicated aprevious study on affordance perception in which participants actively moved through differently wide doors in an immersive 3D virtual environment. In the present study, wedisplayed the same environmenton a 2D screen with participants moving through doors of different widths using the keys on a standard keyboard. We compared components of the event-related potential (ERP) from the continuously recorded electroencephalogram (EEG) that were previously reported to be related to affordance perception of architectural transitions (passable and impassable doors). Comparing early sensory and later motor-related ERPs, our study replicated ERPs reflecting early affordance perception but found differences in later motor-related components. These results indicate a shift from automated perception of affordances during early sensorimotor time windows to movement context dependence of affordance perception at later stages, suggesting that affordance perception is a dynamic and flexible process that changes over sensorimotor stages.

The Reconstructive Metaverse - Collaboration in Real-Time Shared Mixed Reality Environments for Microsurgical Reconstruction.

Plastic surgeons routinely use 3D-models in their clinical practice, from 3D-photography and surface imaging to 3D-segmentations from radiological scans. However, these models continue to be viewed on flattened 2D screens that do not enable an intuitive understanding of 3D-relationships and cause challenges regarding collaboration with colleagues. The Metaverse has been proposed as a new age of applications building on modern Mixed Reality headset technology that allows remote collaboration on virtual 3D-models in a shared physical-virtual space in real-time. We demonstrate the first use of the Metaverse in the context of reconstructive surgery, focusing on preoperative planning discussions and trainee education. Using a HoloLens headset with the Microsoft Mesh application, we performed planning sessions for 4 DIEP-flaps in our reconstructive metaverse on virtual patient-models segmented from routine CT angiography. In these sessions, surgeons discuss perforator anatomy and perforator selection strategies whilst comprehensively assessing the respective models. We demonstrate the workflow for a one-on-one interaction between an attending surgeon and a trainee in a video featuring both viewpoints as seen through the headset. We believe the Metaverse will provide novel opportunities to use the 3D-models that are already created in everyday plastic surgery practice in a more collaborative, immersive, accessible, and educational manner.

Screening of low-friction two-dimensional materials from high-throughput calculations using lubricating figure of merit

Two-dimensional materials are excellent lubricants with inherent advantages. However, superlubricity has been reported for only a few of these materials. Unfortunately, other promising two-dimentional (2D) materials with different physical properties cannot be discovered or applied in production; thus, energy consumption can be greatly reduced. Here, we carry out high-throughput calculations for 1,475 2D materials and screen for low-friction materials. To set a standard, we propose, for the first time, a geometry-independent lubricating figure of merit based on the conditions for stick-slip transition and our theory of Moiré friction. For the efficient calculation of this figure of merit, an innovative approach was developed based on an improved registry index model. Through calculations, 340 materials were found to have a figure of merit lower than 10−3. Eventually, a small set of 21 materials with a figure of merit lower than 10−4 were screened out. These materials can provide diverse choices for various applications. In addition, the efficient computational approach demonstrated in this work can be used to study other stacking-dependent properties.

AdaPIP: Adaptive picture-in-picture guidance for 360° film watching

Abstract360° videos enable viewers to watch freely from different directions but inevitably prevent them from perceiving all the helpful information. To mitigate this problem, picture-in-picture (PIP) guidance was proposed using preview windows to show regions of interest (ROIs) outside the current view range. We identify several drawbacks of this representation and propose a new method for 360° film watching called AdaPIP. AdaPIP enhances traditional PIP by adaptively arranging preview windows with changeable view ranges and sizes. In addition, AdaPIP incorporates the advantage of arrow-based guidance by presenting circular windows with arrows attached to them to help users locate the corresponding ROIs more efficiently. We also adapted AdaPIP and Outside-In to HMD-based immersive virtual reality environments to demonstrate the usability of PIP-guided approaches beyond 2D screens. Comprehensive user experiments on 2D screens, as well as in VR environments, indicate that AdaPIP is superior to alternative methods in terms of visual experiences while maintaining a comparable degree of immersion.

Optimizing Virtual Nature for Psychological and Physiological Well-Being: A Systematic Review of the Moderating Effects of Duration, Nature Type, Sample Characteristics, and Immersiveness and Potential Risks of Bias

Virtual nature research has emerged as a prominent and captivating study area, gaining much attention for its profound potential to enhance well-being. This literature review aimed to expand prior reviews of virtual nature experiences on psychological and physiological well-being in two ways: summarizing how four factors may moderate the beneficial effects of virtual nature and reporting the risk of bias in this body of literature. Searches for peer-reviewed research articles were conducted in Web of Science and Scopus and manually identified, returning 78 relevant empirical studies published between 2010 and 2023. The assessment of bias was conducted utilizing Cochrane’s RoB 2 and ROBINS-I tools. The four moderators examined were duration of exposure (i.e., ≤5 min, 5–10 min, ≥10 min), type of virtual nature (i.e., green space, blue space), sample characteristics (i.e., age, health status), and immersion level (i.e., virtual reality [VR], 2D screens). We found limited evidence for the impact of the first three moderators but stronger evidence for higher levels of immersion showing stronger benefits. All studies were found to have a moderate to high risk of bias, mostly related to the subjective measurement of outcomes. Future research should prioritize studying tailored virtual nature interventions and their long-term effects among diverse participants and different types of virtual environments, as well as investigating the influence of presence and immersion levels in virtual settings. These efforts will provide further insights into the underlying mechanisms of the benefits derived from virtual nature exposure.

Remote practical instruction using web browsers

<p>This paper introduces a novel approach to remote coaching, specifically targeting the body movements of learners participating remotely. The proposed system employs a smartphone camera to capture the learner’s body and represent it as a 3D avatar. The instructor can then offer guidance and instruction by manipulating the 3D avatar’s shape, which is displayed on a web browser. The main challenge faced by the system is to enable the sharing and editing of 3D objects among users. Since the HTML5 drag-and-drop feature is inadequate for transforming virtual objects consisting of multiple interconnected rigid bodies, the system tracks the pivot point of the manipulated rigid body. It assigns attributes such as pivot points and action points to each object, extending beyond their 2D screen coordinates. To implement the system, an interactive web application framework following the model-view-view-model (MVVM) architecture is utilized, incorporating Vue.js, Three.js, and Google Firebase. The prototype system takes advantage of the data binding capability of the framework and successfully operates within the 3D space of a web browser. Experimental results demonstrate that it can effectively share transformation information with an average delay of 300 ms.</p>

Feasibility and accuracy of a real-time depth-based markerless navigation method for hologram-guided surgery

BackgroundTwo-dimensional (2D) medical visualization techniques are often insufficient for displaying complex, three-dimensional (3D) anatomical structures. Moreover, the visualization of medical data on a 2D screen during surgery is undesirable, because it requires a surgeon to continuously switch focus. This switching focus problem also results in extensive use of intraoperative radiation to gain additional insights for a 3D configuration. The use of augmented reality (AR) has the potential to overcome these problems, for instance by using markers on target points that are aligned with the AR solution. However, placing markers for a precise virtual overlay are time-costly, always have to be visible within the field of view and disrupt the surgical workflow. In this study, we developed ARCUS, a depth-based, markerless AR navigation system, which overlays 3D virtual elements onto target body parts to overcome the limitations of 2D medical visualization techniques.Methods and resultsIn a phantom study, our markerless ARCUS system was evaluated for accuracy and precision by comparing it to a Quick Response (QR) code-based AR registration method. The evaluation involved measuring the Euclidean distance between target points on a 3D-printed face and their corresponding points on the virtual overlay using a robotic arm for precise measurements. Correlations between the measuring points provided by our markerless system and the actual measuring points on the 3D-print were high, with promising consistent Euclidean distances between the 3D points and the virtual points generated by both our markerless system and the Vuforia QR Code system. We also show two clinical examples of ex vivo case studies on cadaveric human specimens where our markerless ARCUS system could be applicable to.ConclusionThe markerless AR navigation system holds strong potential as a 3D visualization method in clinical settings. While both ARCUS and the Vuforia QR code-based method fell short of meeting the surgical threshold of a 2 mm offset, our markerless system demonstrated promising features such as instant registration, markerless operation, and potential compatibility with non-rigid structures. Its automated virtual overlay onto target body parts offers significant advantages, paving the way for investigations into future clinical use.

An efficient aluminum gradient mesh wick for enhancing boiling heat transfer performance

With the rapid development of high-power electronic devices, phase change heat transfer has attracted much attention for efficient and lightweight thermal management. However, the mesh structure design for aluminum-based pool boiling enhancement still lacks experimental investigation and theoretical analysis. Herein, an aluminum gradient mesh wick is developed for efficient heat transfer performance. The optimized gradient mesh wick is comprised of a large-hole rhombus flat punching structure on the top and a small-hole 2D screen woven mesh on the bottom, providing sufficient active nucleation density sites, timely liquid supply, and rapid bubble departure. The gradient mesh design for pool boiling enables to achieve a high critical heat flux (CHF) of 1095.46 kW/m2 and a high heat transfer coefficient (HTC) of 46.20 kW/m2·K at a wall superheat of 23.70 °C, which is 167.06% and 240% that of Al plate, respectively. The visualization of bubble behaviors and theoretical analysis reveals that the CHF of the gradient mesh wick mainly depends on the bubble departure and liquid supply, and the bubble departure accounts for a greater proportion. Such aluminum gradient mesh wick provides a strategy for efficient thermal management of lightweight and high-power electronic devices.

Memory Recall for Data Visualizations in Mixed Reality, Virtual Reality, 3D and 2D.

This article explores how the ability to recall information in data visualizations depends on the presentation technology. Participants viewed 10 Isotype visualizations on a 2D screen, in 3D, in Virtual Reality (VR) and in Mixed Reality (MR). To provide a fair comparison between the three 3D conditions, we used LIDAR to capture the details of the physical rooms, and used this information to create our textured 3D models. For all environments, we measured the number of visualizations recalled and their order (2D) or spatial location (3D, VR, MR). We also measured the number of syntactic and semantic features recalled. Results of our study show increased recall and greater richness of data understanding in the MR condition. Not only did participants recall more visualizations and ordinal/spatial positions in MR, but they also remembered more details about graph axes and data mappings, and more information about the shape of the data. We discuss how differences in the spatial and kinesthetic cues provided in these different environments could contribute to these results, and reasons why we did not observe comparable performance in the 3D and VR conditions.

Research on the design and presentation of traditional ethnic costumes based on virtual reality technology

Abstract In this paper, the 3D model of the human body is drawn, and the 3D transformation in the 3D garment piece design is realized by using the inverse mapping of 2D screen coordinates to 3D model coordinates. Then, the technique of virtual presentation of traditional ethnic clothing is studied, alternating the left and in-eye views and using the overlap of calculation to optimize the calculation to achieve a three-dimensional display. The virtual design and presentation effects of traditional ethnic costumes were assessed through a comprehensive comparison, sensory experience, interactive experience, emotional experience, and cognitive experience. The results showed that the regression coefficients of cognitive experience, sensory experience, interactive experience, and emotional experience of the virtual design and presentation of costumes were 0.98, 0.875, 0.993, and 0.996, respectively, which were 0.506, 0.36, 0.11, 0.161, and 0.048 more than the regression coefficients of the traditional display method in each dimension. This study provides the theoretical basis for combining traditional costumes with virtual technology.

X-Space: Interaction design of extending mixed reality space from Web2D visualization

Mixed reality offers a larger visualization space and more intuitive means of interaction for data exploration, and many works have been dedicated to combining 2D visualizations on screen with mixe reality. However, for each combination, we need to customize the implementation of the corresponding mixed reality 3D visualization. It is a challenge to simplify this development process and enable agile building of mixed reality 3D visualizations for 2D visualizations. In addition, many existing 2D visualizations do not provide interfaces oriented to immersive analytics, so how to extend the mixed reality 3D space from existing 2D visualizations is another challenge. This work presents an agile and flexible approach to interactively transfer visualizations from 2D screens to mixed reality 3D spaces. We designed an interactive process for spatial generation of mixed-reality 3D visualizations, defined a unified data transfer framework, integrated data deconstruction techniques for 2D visualizations, implemented interfaces to immersive visualization building tool-kits, and encapsulated these techniques into a tool named X-Space. We validated that the approach is feasible and effective through 2D visualization cases including scatter plots, stacked bar charts, and adjacency matrix. Finally, we conducted expert interviews to discuss the usability and value of the method.

RTO 사고 예방을 위한 메타버스 기반 안전 서비스 플랫폼

This paper implements a Metaverse-based safe operation service platform for safe maintenance and operation of RTO. A Regenerative Thermal Oxidation (RTO) is a device that prevents volatile organic compounds (VOCs) gas generated from chemical plants, paint production and coating manufacturing, and metal surface treatment from being discharged into the atmosphere. If the plant is operated 24 hours a day, the RTO also continues to operate. In order to operate safely and preserve accident prediction, guidance and safety education on equipment are necessary.
 Most of the 24-hour RTOs are long-distance, so if the equipment fails, there is a limit to immediate response, and an inefficient maintenance situation occurs. In particular, long-distance monitoring and safety training for operation are needed due to the immature handover caused by the aging trend of field personnel, complex equipment use, and personnel turnover.
 A virtual space is constructed based on a metaverse, and an assembly manual for equipment structure, circuit diagrams, component characteristics, and detailed composition is shaped in 3D using digital twin technology. To this end, 3D modeling technology generates a mesh based on the design data of the RTO equipment and generates a texture of the equipment based on the actual picture. Through the reconstructed 3D virtual space in the field for 3D virtual space configuration, the user's parts are tracked by delivering immersive low latency and high-quality data to remote XR devices, PCs, and smartphones based on multi-user. At this time, AR glass (smartphone) is used to share the location of the remote user with the site as location data.
 Most studies on the safe operation of RTO devices have focused on hardware safety through field monitoring and data acquisition for pressure, temperature, and VOCs emissions through sensor IoT. Most of the plants are located at a long distance, and the time to stop the plant for repair and repair work is limited. Accordingly, monitoring, which was previously operated as a 2D screen, is combined with 3D space to provide an intuitive and visible environment and efficiency.
 By predicting the failure state of the facility through metaverse-based monitoring, the efficiency of maintenance can be improved, and productivity and market competitiveness can be secured by minimizing downtime. In addition, education and training in virtual space reduces the risk of failure and maintains a high level of facility maintenance to establish a safe working environment. Therefore, the system's overall safety platform can cope with serious disasters and minimize environmental impacts such as gas leaks as well as equipment failures.

Harnessing the power of virtual reality: Enhancing telepresence and inspiring sustainable travel intentions in the tourism industry

This study investigates how virtual reality (VR) can enhance telepresence and stimulate VR use and travel intentions in the sustainable tourism industry. 512 local and international tourists in Vietnam participated in an online survey after experiencing a VR experiment. They interacted with an app presenting a sustainable tourist destination based on nature, with the treatment group using VR sets and the control group viewing it on a 2D screen. Results indicate that VR-based tours facilitate greater telepresence than 2D presentations. VR is not a substitute for actual travel, with higher intentions reported to use VR in the future and to visit the tourist destination in real life due to their greater telepresence and perceived enjoyment from VR. This has implications for nature-based sustainable tourism, with further research needed for other types of tourism, including extreme activities.

Kinetic model of vibration screening for granular materials based on biological neural network

The kinetic model is the theoretical basis for optimizing the structure and operation performance of vibration screening devices. In this paper, a biological neurodynamic equation and neural connections were established according to the motion and interaction properties of the material under vibration excitation. The material feeding to the screen and the material passing through apertures were considered as excitatory and inhibitory inputs, respectively, and the generated stable neural activity landscape was used to describe the material distribution on the 2D screen surface. The dynamic process of material vibration screening was simulated using discrete element method (DEM). By comparing the similarity between the material distribution established using biological neural network (BNN) and that obtained using DEM simulation, the optimum coefficients of BNN model under a certain screening parameter were determined, that is, one relationship between the BNN model coefficients and the screening operation parameters was established. Different screening parameters were randomly selected, and the corresponding relationships were established as a database. Then, with straw/grain ratio, aperture diameter, inclination angle, vibration strength in normal and tangential directions as inputs, five independent adaptive neuro-fuzzy inference systems (ANFIS) were established to predict the optimum BNN model coefficients, respectively. The training results indicated that ANFIS models had good stability and accuracy. The flexibility and adaptability of the proposed BNN method was demonstrated by modeling material distribution under complex feeding conditions such as multiple regions and non-uniform rate.

Direct comparison of virtual reality and 2D delivery on sense of presence, emotional and physiological outcome measures

Introduction: Virtual-reality (VR) technology has, over the last decade, quickly expanded from gaming into other sectors including training, education, and wellness. One of the most popular justifications for the use of VR over 2D is increased immersion and engagement. However, very little fundamental research has been produced evaluating the comparative impact of immersive VR on the user’s cognitive, physiological, and emotional state.Methods: A within-subject cross-over study design was used to directly compare VR and 2D screen delivery of different subject matter content. Both physiological and self-report data were collected for scenes containing calming nature environments, aggressive social confrontations, and neutral content.Results: Compared to 2D, the VR delivery resulted in a higher sense of presence, higher ratings of engagement, fun, and privacy. Confrontational scenes were rated as more tense whilst calming scenes were rated as more relaxing when presented in VR compared to 2D. Physiological data indicated that the scenes promoted overall states of arousal and relaxation in accordance with the scene subject matter (both VR and 2D). However, heart rate (HR) and galvanic skin response (GSR) were consistently higher throughout the VR delivery condition compared to 2D, including responses during scenes of neutral and calming subject matter.Discussion: This discrepancy between emotional and physiological responses for calming and neutral content in VR suggest an elevated arousal response driven by VR immersion that is independent of the emotional and physiological responses to the subject matter itself. These findings have important implications for those looking to develop and utilize VR technology as a training and educational tool as they provide insights into the impact of immersion on the user.

Challenges of 3D Surface Reconstruction in Capsule Endoscopy.

Essential for improving the accuracy and reliability of bowel cancer screening, three-dimensional (3D) surface reconstruction using capsule endoscopy (CE) images remains challenging due to CE hardware and software limitations. This report generally focuses on challenges associated with 3D visualization and specifically investigates the impact of the indeterminate selection of the angle of the line-of-sight on 3D surfaces. Furthermore, it demonstrates that impact through 3D surfaces viewed at the same azimuth angles and different elevation angles of the line-of-sight. The report concludes that 3D printing of reconstructed 3D surfaces can potentially overcome line-of-sight indeterminate selection and 2D screen visual restriction-related errors.

Effects of Virtual Reality on Complex Building System Recall

Multidisciplinary design and construction teams are challenged to communicate and coordinate across complex building systems, including architectural, structural, mechanical, electrical, and piping (MEP). To support this coordination, disciplinary 3D models are combined and coordinated before installation. Studies show that besides the use of 3D models, industry professionals sketch building components to discuss coordination issues and find resolutions that require them to recall the building components in the model. In current practices, 3D models are explored with Building Information Modeling (BIM) tools presented on 2D screens, while Virtual Reality (VR) can provide users with an immersive environment to explore. This paper presents the results of an experiment that studied the effects of VR’s immersive environment on the participants’ complex MEP system recall compared to BIM via sketching. The comparison criteria were the 3D geometry properties of the piping system and the users’ self-awareness in the model categorized under color, shape, dimension, piping, and viewpoint. The results showed significant improvement in recall of shape, dimension, and piping when the model was explored in VR.