Immersive Scene Research Articles

Immersive representation for visualizing surface temperature and integrative lighting in architecture

This paper illustrates the potential of representing and reading environmental data captured for immersive visualization of indoor built environments. This study introduces methods for qualitative representation of light and heat in aligned viewing directions in immersive scenes. It employs a novel technique for capturing High Dynamic Range (HDR) and thermal images in 360-degree directions. This technique allows visualization of radiant temperature and integrative lighting variations associated with the viewing direction and time. The highlight of these visualizations is the identical viewing points in studying the indoor ambiances. The paper provides visualizations derived from hourly data captures in architectural studios through a 360-degree thermal-visual capturing device. The paper aims to present these results through various visualization formats, including architectural planar presentations, diagrams, and colour maps, showing surface temperature and Photopic and Melanopic illuminance intensities. Architectural insights gained from each of these visualizations are highlighted and discussed. Specifically, the results of this research illustrate the potential of the immersive image-based environmental assessment and visualization method for representing indoor ambiances regarding their non-visible thermal and lighting conditions through time and across all viewing directions within the space.

Immersive scene representation in human visual cortex with ultra-wide-angle neuroimaging

While human vision spans 220°, traditional functional MRI setups display images only up to central 10-15°. Thus, it remains unknown how the brain represents a scene perceived across the full visual field. Here, we introduce a method for ultra-wide angle display and probe signatures of immersive scene representation. An unobstructed view of 175° is achieved by bouncing the projected image off angled-mirrors onto a custom-built curved screen. To avoid perceptual distortion, scenes are created with wide field-of-view from custom virtual environments. We find that immersive scene representation drives medial cortex with far-peripheral preferences, but shows minimal modulation in classic scene regions. Further, scene and face-selective regions maintain their content preferences even with extreme far-periphery stimulation, highlighting that not all far-peripheral information is automatically integrated into scene regions computations. This work provides clarifying evidence on content vs. peripheral preferences in scene representation and opens new avenues to research immersive vision.

Research on the Innovative Development of Digital Media Enabled by Computer Visual Arts

Abstract This paper explores the integration of computer visual art with digital media, highlighting the innovative potential of computer technology in creating dynamic digital media art. Initially, we review relevant theories of computer-based visual art and examine their applications in digital media art. We then detail an immersive scene design using Web3D technology, where images are processed via a computer vision camera imaging model, transformed, and analyzed with the SIFT algorithm to extract feature points. These are integrated with the incremental SFM algorithm for three-dimensional image reconstruction in immersive environments. We validate these methods with real datasets and controlled experiments to assess user experience improvements in digital media applications. Results indicate that the SIFT algorithm’s image-matching accuracy ranges from 50% to 78%, and the incremental SFM algorithm shows an average rotation error of only 0.183°. User sentiment scores improved significantly, from 11.583 to 23.452, post-experience, demonstrating enhanced realism and a new direction for digital media art.

Integrating Immersive Scenes and Interactive Contextual Clue Scaffolding Into Decision-Making Analysis Ability Training Game

In the post-pandemic era, distance education and training modes without space constraints are gradually becoming a new trend in training. Game-based training combined with situational simulations is a potential approach. This study used a simulation company scenario and a decision-making analysis task of promotion assessment as the game context, and proposed an innovative game mechanism integrating the simulation situation scenario and the interactive contextual clue scaffolding as a guide to enhance learning effectiveness. This study used the Gather Town platform to design a simulation office scenario, and Google Forms to design interactive contextual clue scaffolding. This study conducted empirical research to evaluate this mechanism, with a total of 60 participants divided into an experimental group that conducted game activities and a control group that was only provided with document files as scaffolding for comparative analysis. The results of the study revealed that the proposed exploration guidance mechanism of integrating immersive scenes and interactive contextual clue scaffolding could allow learners to experience a high level of game fidelity and increase the discovery rate and usefulness of the scaffolding. It also reduced learning anxiety, maintained learning motivation and high flow, and resulted in significantly higher learning effectiveness than the control group approach.

Immersive experience of Chinese poems and paintings

Chinese painting incorporated with poems is famous for its distinct characteristics and unspeakable moods. Ink lines are used to depict scenes in paintings while concise words are used to describe scenes and events in poems, making for a beautiful yet enigmatic art style. Such an art style requires viewers to patiently examine the details and attempt to understand the meaning, which creates a barrier for a wide audience to relish and disseminate it. The condition can be effectively improved with immersive and interactive animation techniques. In this paper, we present the design and production of an interactive 3D animation entitled "Illustrating Ten Poems" based on the famous Chinese painting with the same name. Compared with viewing the original painting, the animation provides immersive scene reconstruction in Chinese painting style, interactive characters and visual-audio experience. Rather than complete three-dimensional reconstruction, we attempted some innovative methods which proved to be effective and efficient in Chinese painting style artwork production.

Research on Marketing Construction of Xiamen's "Natural Studio" Scene —— based on AIDEES model

Xiamen's "natural studio" scene construction experience is inexperienced, and there are some problems, such as scattered film and television tourism resources, uneven distribution of film and television tourism resources, weak linkage between film and television scenes and tourism scenes, less science and technology implantation scenes, lack of directional talents, and so on. In order to construct Xiamen "natural studio" scene marketing more accurately and effectively, Xiamen can draw lessons from Disney's scene marketing experience in space dimension, communication dimension, consumer perspective, scene service and technical means based on AIDEES model, with consumers as the core in scene marketing, create Xiamen style propaganda channel to attract attention, carry out consumption campaign to cultivate interest, and create solid scene backing to stimulate desire. Create immersive scenes to focus on consumer experience; improve the scene chain to continue consumer enthusiasm and build online sharing scenes to complete consumer personal sharing.

Evaluating Flight Performance and Eye Movement Patterns Using Virtual Reality Flight Simulator

Efficient and economical performance evaluation of pilots has become critical to the aviation industry. With the development of virtual reality (VR) and the combination of eye-tracking technology, solutions to meet these needs are becoming a reality. Previous studies have explored VR-based flight simulators, focusing mainly on technology validation and flight training. The current study developed a new VR flight simulator to evaluate pilots' flight performance based on eye movement and flight indicators in a 3D immersive scene. During the experiment, 46 participants were recruited: 23 professional pilots and 23 college students without flight experience. The experiment results showed significant differences in flight performance between participants with and without flight experience, the former being higher than the latter. In contrast, those with flight experience showed more structured and efficient eye-movement patterns. These results of the differentiation of flight performance demonstrate the validity of the current VR flight simulator as a flight performance assessment method. The different eye-movement patterns with flight experience provide the basis for future flight selection. However, this VR-based flight simulator has shortcomings like motion feedback compared to traditional flight simulators. This flight simulator platform is highly flexible except for the apparent low cost. It can meet the diverse needs of researchers (e.g., measuring situation awareness, VR sickness, and workload by adding relevant scales).

QoE-Driven Adaptive Streaming for Point Clouds

With increasing popularity of virtual reality and augmented reality, application of point clouds is in critical demand as it enables users to freely navigate in an immersive scene with six degrees of freedom. However, point clouds usually comprise large amounts of data, and are thus difficult to stream in bandwidth-constrained networks. It is therefore important, yet challenging, to efficiently stream the resource-intensive point clouds, such that the user's quality of experience (QoE) is guaranteed on a high-level but with a low bandwidth consumption. To this end, we propose a QoE-driven adaptive streaming approach for the tile-based point cloud transmission, to maximize the user's QoE while reducing the transmission redundancy. By exploiting the perspective projection, we specifically model the QoE of a 3D tile as a function of the bitrate of its representation, user's view frustum and spatial position, occlusion between tiles, and the resolution of rendering device. Based on this QoE model, we then formulate the QoE-optimized rate adaptation problem as a multiple-choice knapsack problem, which allocates bitrates for different tiles under a given transmission capacity. It is equivalently converted to a submodular function maximization problem subject to knapsack constraints, and solved by a practical greedy-based algorithm with a theoretical worst-case performance guarantee. The proposed algorithm is able to achieve a near-optimal performance, but with a very low computational complexity. Experimental results further demonstrate superiority of the proposed rate adaptation algorithm over existing schemes, in terms of both user's visual quality and transmission efficiency.

News products from the perspective of metaverse: Social motivation, practical model and conceptual change

<p>News products from the perspective of metaverse are the result of the interaction between technology, news production and people’s needs, which turns news consumption from the information consumption of the audience’s real identity to the immersive scene consumption of digital separation. This paper summarizes four practical models of news products from the perspective of metaverse, analyzes the innovation of traditional news value concept by metaverse technology, and makes a cold reflection on news products from the perspective of metaverse.</p>

News products from the perspective of metaverse: Social motivation, practical model and conceptual change

<p>News products from the perspective of metaverse are the result of the interaction between technology, news production and people’s needs, which turns news consumption from the information consumption of the audience’s real identity to the immersive scene consumption of digital separation. This paper summarizes four practical models of news products from the perspective of metaverse, analyzes the innovation of traditional news value concept by metaverse technology, and makes a cold reflection on news products from the perspective of metaverse.</p>

The Integration Mechanism of Music Intangible Cultural Heritage and Tourism Industry in the Internet of Things Environment

This work aims to optimize and innovate the protection, inheritance, and dissemination of Intangible Cultural Heritage (ICH). Based on the Internet of Things (IoT), this work designs the music ICH development path under the cultural concept by integrating music ICH with the tourism industry in the smart city environment. Specifically, it innovatively proposes the integration mechanism of immersive scene design of music ICH with smart city’s tourism industry in the Internet of Things (IoT) context. Consequently, the music ICH brand equity evaluation model is built. This model can evaluate the importance of cultural and asset value of music ICH. The comprehensive evaluation index system of music ICH mainly includes four primary indexes (with eight secondary indexes): the live performance, historical value, music audience, and the tourism industry integration. In addition, at the beginning of the model iteration, the pattern recognition accuracy of the traditional model is only 67.5%. Then, the recognition accuracy of the traditional model can reach 72.5% at the 500th iteration. In contrast, the average recognition accuracy of the improved model can reach 80%, and the highest can reach 82.5%. This work can provide technical support and guarantee for formulating and optimizing music ICH protection strategies.

The Influence of Immersive Virtual Reality (IVR) on Skill Transfer of Learners: The Moderating Effects of Learning Engagement

Immersive virtual reality (IVR) is a full integration of virtual simulation and reality through computer engines and digital technologies. In particular, upon entering the field of education, IVR triggers the comprehensive reform of the education industry, realizes the interaction between virtual and real worlds, shapes a more highly immersive teaching scene, and meets the personalized needs of different learners. Thus, exploring the transfer of learning skills under IVR and traditional teaching modes is of great significance to understanding their causal relationship. Results show that training methods have significant differences in post-test scores. Immersive virtual experimental teaching plays a significant role in the skills transfer of learners, and learning engagement has a significant moderating effect in the above relationship. In this study, the results have important reference value for enriching the design of immersive teaching content, improving its design of translation, and promoting the skills transfer of learners to higher-order thinking knowledge.

Research on Immersion Teaching Method Based on 5G +XR Technology and Reinforcement Learning Model

The traditional teaching mode is difficult to meet the needs of modern teaching development because of its single teaching method and insufficient utilization of resources. With the in-depth application of 5G and XR technology in the field of education, the teaching mode based on 5G + XR has become a research hotspot of educational informatization. Therefore, this paper proposed an immersive teaching method based on 5G + XR technology and reinforcement learning model. By analyzing the characteristics and advantages of 5G technology in the field of teaching, this paper expounded that 5G technology not only had an important impact on the reform of modern teaching mode, but also can promote the development of immersive teaching. In addition, based on the analysis of XR teaching theory and its significance, an immersive XR teaching design method based on reinforcement learning model was proposed, and the working principle of immersive XR teaching system and the architecture method of teaching space were given. Finally, the construction scheme of immersive teaching scene based on 5G + XR and the design idea of immersive XR teaching method were put forward. The experiment showed that the immersive XR teaching method proposed in this paper was better than the traditional teaching mode, whether it was the students' learning effect of knowledge or their participation and experience in learning, which also verified that the method proposed in this paper had certain feasibility and superiority. The immersion teaching method based on 5G + XR proposed in this paper can provide a reference for the reform and development of modern education and teaching mode.

Virtual and Augmented Reality Use Cases for Fusion Design Engineering

As augmented and virtual reality technology is revolutionising the way products are designed and tested, many companies are expanding the range of use cases across their organisations. After 10 years using Virtual Reality (VR) & Augmented Reality (AR) technology during the life cycle of fusion components, the CEA IRFM has improved its skills developing immersive visualisation scenes for the WEST tokamak [6, 5] project and several ITER Site Support Agreement such as Magnet Infrastructure Facilities for ITER (MIFI) [7] and Test Blanket Module System (TBM) [8]. Data preparation process has been optimized, reducing time cost to convert CAD data to an interactive and immersive model. For the WEST Project, VR engineers developed immersive scenes to simulate assembly sequences, to check component introduction using physics and to show 2D temperature data from diagnostics in a 3D environment. For the TBM case, health and safety oriented model environment is developed. Features for team simulated human access and hands on operations for As Low As Reasonably Achievable (ALARA) optimisation process with motion-tracking system and virtual avatars with ventilated protective pressurised suit simulating worker protection in nuclear environment. For MIFI application, a specific assembly tool has been developed for TF coils interface located on the top and at the bottom junction. During a first phase, the development was driven by iterative design loop (CAD modelling analysis and cinematic by using VR simulations). By means of interactive simulations and immersive conditions, the interface connection task was validated against feasibility including accessibility. AR has also been introduced in the process since a physical mock-up has been augmented to consider the constraint environment of TF Coils. For TBM application, on site AR simulation of the port Cell equipment will provide enhanced environment description. Collaborative immersive context, digital twins and combination between AR/VR & physical mock-ups for validation of maintenance since component design phase will constitute challenge for the near future.

Design and Research of Interactive Animation of Immersive Space Scene Based on Computer Vision Technology

With the development of computational simulation technology, the need for stable and immersive display effects of space scene animation in the field of life experience and visual art has gradually increased. In this case, the requirements for immersive characteristics of space scene animation have also been strengthened. The existing 3D space scene animation has a limited degree of stereoscopic display model and data visualization. However, the current space scene interactive animation adopts a plane layout as a whole, and the size of the view interface is generally fixed, including the size of virtual elements. However, the immersion in this paper interactive animation of spatial scene can effectively solve the problems of incomplete display of 3D effects and unstable view simulation in 3D effects. This paper takes immersive space scene animation as the research object and studies the 3D and characteristics of space scene animation based on computer parallel computing in different immersive space scenes and different virtual space technologies, as well as the animation effects of different scene transformations and art forms. The result of research shows that, with the continuous increase of the degree of virtuality within a certain range, the immersive effect of spatial scene interactive animation gradually becomes better. When the color of the space scene animation is below 15, the virtual immersive effect changes less. When the space scene is in the range of 15–20, the space scene will make people feel the atmosphere of a beautiful and mysterious illusion.

Intuitive robot teleoperation for civil engineering operations with virtual reality and deep learning scene reconstruction

Robotic teleoperation, i.e., manipulating remote robotic systems at a distance, has gained its popularity in various industrial applications, including construction operations. The key to a successful teleoperation robot system is the delicate design of the human-robot interface that helps strengthen the human operator’s situational awareness. Traditional human-robot interface for robotic teleoperation is usually based on imagery data (e.g., video streaming), causing the limited field of view (FOV) and increased cognitive burden for processing additional spatial information. As a result, 3D scene reconstruction methods based on point cloud models captured by scanning technologies (e.g., depth camera and LiDAR) have been explored to provide immersive and intuitive feedback to the human operator. Despite the added benefits of applying reconstructed 3D scenes in telerobotic systems, challenges still present. Most 3D reconstruction methods utilize raw point cloud data due to the difficulty of real-time model rendering. The significant size of point cloud data makes the processing and transfer between robots and human operators difficult and slow. In addition, most reconstructed point cloud models do not contain physical properties such as weight and colliders. A more enriched control mechanism based on physics engine simulations is impossible. This paper presents an intelligent robot teleoperation interface that collects, processes, transfers, and reconstructs the immersive scene model of the workspace in Virtual Reality (VR) and enables intuitive robot controls accordingly. The proposed system, Telerobotic Operation based on Auto-reconstructed Remote Scene (TOARS), utilizes a deep learning algorithm to automatically detect objects in the captured scene, along with their physical properties, based on the point cloud data. The processed information is then transferred to the game engine where rendered virtual objects replace the original point cloud models in the VR environment. TOARS is expected to significantly improve the efficiency of 3D scene reconstruction and situational awareness of human operators in robotic teleoperation.

Depth-Guided Pixel Dimming With Saliency-Oriented Power-Saving Transformation for Stereoscope AMOLED Displays

With better experience of immersive scene, the virtual reality (VR) becomes the main stream of entertainments. Featuring high chromatic displaying performance, relatively higher efficiency in power and lower latency, the Active-Matrix Organic Lighting-Emitting Diode (AMOLED) is the best-choice display technology for VR. However, AMOLED still requires the higher power consumption amongst other components on smart phones. Thus, various studies attempted to leverage the property of AMOLED to lower the power consumption while preserving the visual quality. This paper proposes a perceptually saliency-oriented transformation by incorporating the concept of how human eyes work in VR environment to provide more suitable experience while saving more power consumption. Experimental results show the proposed method preserves up to 49.33% of the power demand with high perceptual quality as other existing techniques. Furthermore, the subjective experiment demonstrates the proposed method provides a better perceptual quality for VR environment.

Immersive journalism: the factors of effective functioning

The main approaches to the definition of the “immersive journalism” phenomenon is considered and its working definition is derived. This working definition incorporates both traditional and newest approaches to the structure-forming, technological and functional factors of the production of VR-content. There are the levels of immersion into the story are analyzed on the example of video–360 ° scenes (posted on the official YouTube channel of t The New York Times) in this study. The factors influencing the formation of the system of user preferences are studied. The results of vidIQ analysis of five the most popular immersive scenes are compared and presented. It tried to measure the presence in the virtual environment by the of the Witmer-Singer’s method. The study also used the methodology of actor-network theory and the approaches of Maria-Laure Ryan. The object of study are the most popular vidIQ assessment immersive video of “The New York Times” (2015–2017): “Walking New York”, “The Atomic Bombing of Hiroshima”, “The Fight for Falluja”, “Seeking Pluto's Frigid Heart” and “The Displaced” as the one of the most resonant immersive publications. In this empirical material, the components of the VR effect are highlighted: presence, involvement, immersion. Each of the components is built up by the functioning of a few factors from the Witmer-Singer model. It’s determined that the components of the VR effect are not equivalent. The basis of the immersive narration is the effect of presence, supported either by immersion in the storu, or by involvement into the environment. The results indicate that it’s quite difficult to consider the whole complex of factors in the production of journalistic materials. In full, they work in making and consuming of not immersive, but VR-content. For the media, the VR technology is not yet a priority, and therefore they prefer to create a presence effect through the possibilities of influencing the algorithm of narrative deployment and the realism presented environment.

GENERATIVE MODELING, VIRTUAL REALITY AND HBIM INTERACTION: IMMERSIVE ENVIRONMENT FOR BUILT HERITAGE: CASE STUDY OF SHAIKH ISA BIN ALI HOUSE, BAHRAIN

Abstract. The innovation in Digital Cultural Heritage (DCH) shows an emergence of innovative methods and initiatives for the generation of complex historical models to which are linked useful information for specialists of architecture, construction and engineering, but also user community interested in cultural heritage. This paper presents an improvement in Historic Building Information Modeling (HBIM) turned into models for mobile apps based on virtual reality (VR) addressed to enhance the communication of Architectural Heritage. The opportunity of a collaboration with the Bahrain Authority for Culture and Antiquities (BACA) experts under the coordination of the Carleton University in a training project aimed at improving documentation skills for conservation and preventive maintenance of built heritage provided the availability of an extraordinary richness of data and surveying on the Shaikh Isa Bin Ali House. The case study of the Shaikh Isa Bin Ali House is an outstanding example of Bahrain architecture and one of the oldest surviving building in Muharraq, listed under the UNESCO sites, now transformed in a Museum, where the visit does not allow to perceive the richness of the traditions of the constructions, beside the beautiful architecture. The building was the seat of the ruler of Bahrain from 1869 until his death in 1932 and represented an interesting response to local building materials and climate. This vernacular architecture solved not only the climate problems but combined the solution with beauty, physical and social functionality. This research carries out digital preservation through HBIM and switches to Cloud and VR platforms to allow users to enter a 3D immersive scene, using models as a vehicle of content information.

KinFit: A Factual Aerobic Sport Game with Stimulation Support

Overweight and obesity is a situation where a person has stacked too much fat that might affect negatively his/her health. Many people skip doing exercises due to several facts related to the encouragement, health-awareness, and time ar-rangement. Diverse aerobic video games have been proposed to help users in do-ing exercises. However, we observe some limitations in existing games. For in-stance, they don’t give correct scores while wearing Arabic traditional suits, they don’t consider showing immersive realistic scenes, and they don’t stimulate users to do exercises and keeping them encouraged to play more. We propose in this paper an aerobic video game that displays real scenes of aerobic coaches and keeps the user notified about doing exercises. It is a kind of serious games that allows users to learn aerobic movements and practice with aerobic coaches. It contains several exercises in which each can be played on normal screen or in fully immersive virtual reality (VR). While the user is playing, he/she can see the playing score with the estimated amount of burned calories. It stores the time when the user plays to remind him/her about doing exercises again. The profound user studies demonstrated the usability and effectiveness of the proposed game.