Immersive Display Research Articles

3‐3: Anamorphic‐XR: Imaging Waveguide Technology for Efficient and Wide Field‐of‐View Near‐Eye Display

A near‐eye display technology, Anamorphic‐XRTM, incorporating imaging waveguides is described. Developed for multiple applications since 2011, unique light manipulation capabilities and highly compact form factors are achievable. Catadioptric anamorphic optics and polarisation management are used to sidestep limitations of conventional VR lens and AR waveguide displays, enhancing performance and manufacturability. Simulations and measurements indicate an achievable specification of 90º field‐of‐view at 60ppd; >2000 nits/lumen; 80% polarised light transmission; and >15mm eyebox at 18mm eye relief from a <4mm thick waveguide. A Hybrid Immersive Display is proposed, combining an anamorphic waveguide with a conventional VR display.

Research on Virtual Simulation Experiment Teaching of Scraper Conveyor Condition Monitoring and Fault Diagnosis

The scraper conveyor status monitoring and fault diagnosis virtual simulation experiment system is designed and developed based on the operating environment of the scraper conveyor in the fully mechanized mining face. Using augmented virtual reality teaching resources, through immersive display and interactive operation experiments, students are guided on basis of scraper conveyor mechanical cognition, master the virtual teaching experiment of state monitoring and fault diagnosis such as broken chain monitoring, motor fault classification and identification, reducer compound fault separation, etc. The teaching practice shows that the system can overcome the harsh production environment in coal mines and the risk factors caused by equipment failure, improve students' understanding of the mechanical fault diagnosis technology, and mobilize students' enthusiasm for experiments and initiative to enhance students' innovative ability.

Hybrid Attention Feature Refinement Network for Lightweight Image Super-Resolution in Metaverse Immersive Display

Hybrid Attention Feature Refinement Network for Lightweight Image Super-Resolution in Metaverse Immersive Display

Innovative Strategies of Virtual Reality Technology in Ethnic Dance Inheritance

Abstract In the quest to safeguard and revitalize folk dance heritage, this study explores the transformative potential of virtual reality (VR) technology. We developed an immersive display system using the Unreal game engine, incorporating a dance movement database, virtual reality immersion, and precise inertial motion capture. This innovative approach captivates users with an immersive folk dance experience and boasts a high motion recognition accuracy (99.51%) and an overall accuracy of 96%. Comparative experiments highlight VR’s advantages in making folk dance education more interactive and enjoyable without sacrificing ease of use. Virtual reality emerges as a vital tool in engaging new generations with folk dance, suggesting a vibrant future for intangible cultural heritage preservation.

MF+C: Linking suboptimal projections to detail on handheld devices

Mobile Focus + Context (mF+C) involves using a handheld device as a focus screen for content on an immersive display or mobile projector. In this work we examine how using a focus device can mitigate poor context image quality due to environmental factors. In an exploratory study we compare three techniques for linking focus and context: lens-focus (Lens), where the device works as a mobile lens held parallel to and in front of the context, center-focus (Centered), where the user holds the device in the center of the projection and pans both context and focus by swiping, and marker-focus (Marker), where the focus region is highlighted on the context, and the user pans the focus by swiping. Participants performed searching, tracing, and detail acquisition tasks with maps and electronics diagrams under a range of simulated projection conditions. All techniques were able to mitigate poor projection quality and performed comparably in time and precision, but the effectiveness of each technique was impacted by task type. There was lower variation in time between participants using Lens for tracing tasks versus the other techniques, but wider variation for searching tasks. Tasks completed using sub-optimal projections involved more time spent looking at the context image than tasks with clear projections, however this difference is less pronounced for the Lens technique. We propose a hybrid Lens-Marker approach for mobile Focus+Context applications in dynamic environments.

Immersive and Challenging Experiences through A Virtual Reality Musical Instruments Game: An Approach to Gamelan Preservation

This study proposes the preservation of traditional Javanese musical instruments known as Gamelan using an immersive and challenging approach in a virtual reality game format. In an effort to achieve a challenging experience, the gamification is designed by observing musical instrument games selected based on their popularity in the Google Play and the App Store. Oculus Quest 2 which is a stand-alone Head Mounted Displays that provides a high-quality immersive display and more flexibility for users to move is chosen to present the game. Overall, the research target can be achieved based on the evaluation using Mean Opinion Scores. The degree of immersion, presence, likeable, challenging, and the value of cultural preservation contained in the game reaches a range of values between good and excellent. Meanwhile, cybersickness still seems to be a chore for developers to make Head Mounted Displays devices more comfortable.

TCCL-Net: Transformer-Convolution Collaborative Learning Network for Omnidirectional Image Super-Resolution

As virtual reality and metaverse become more and more popular, the Omnidirectional Image (OI) has attracted extreme attention due to its immersive display characteristics. However, users only watch a portion of the content in a specific viewport extracted from a panoramic view, which will lead to a problem of resolution mismatch that requires High-Resolution (HR) for clear near-eye displays in viewports. Hence, it is necessary to exploit a Super Resolution (SR) solution for reconstructing Low-Resolution (LR) OIs. Different from 2D SR methods, the variation of pixel distributions along latitudes is a critical factor in designing an Omnidirectional Image Super-Resolution (OISR) scheme. In this paper, we put forward a novel end-to-end network with a Transformer and Convolution Collaborative Learning Network (TCCL-Net) for OISR. Firstly, Swin Transformer blocks and residual convolution blocks are employed to extract long-range and short-range dependencies, thereby digging into more rich and heterogeneous features from these two branches. Secondly, to better fuse these two features, cross-guided enhanced attention mechanisms are designed for bidirectional information enhancement onto both channel and spatial features. Thirdly, to alleviate nonuniformly pixel distributions across latitudes, we add an absolute positional encoding into Swin Transformer to represent patch weights at different positions and propose a tile-based panoramic reconstruction module to super-resolve various bands with different pixel sampling characteristics across latitudes. Experimental results on two available benchmark datasets demonstrate the superiority of the proposed approach over the state-of-the-art method in achieving OISR task.

Unraveling the Design Space of Immersive Analytics: A Systematic Review.

Immersive analytics has emerged as a promising research area, leveraging advances in immersive display technologies and techniques, such as virtual and augmented reality, to facilitate data exploration and decision-making. This paper presents a systematic literature review of 73 studies published between 2013-2022 on immersive analytics systems and visualizations, aiming to identify and categorize the primary dimensions influencing their design. We identified five key dimensions: Academic Theory and Contribution, Immersive Technology, Data, Spatial Presentation, and Visual Presentation. Academic Theory and Contribution assess the motivations behind the works and their theoretical frameworks. Immersive Technology examines the display and input modalities, while Data dimension focuses on dataset types and generation. Spatial Presentation discusses the environment, space, embodiment, and collaboration aspects in IA, and Visual Presentation explores the visual elements, facet and position, and manipulation of views. By examining each dimension individually and cross-referencing them, this review uncovers trends and relationships that help inform the design of immersive systems visualizations. This analysis provides valuable insights for researchers and practitioners, offering guidance in designing future immersive analytics systems and shaping the trajectory of this rapidly evolving field.

Walkable auralizations for experiential learning in an immersive classroom.

This paper proposes an experiential method for learning acoustics and consequences of room design through the rapid creation of audio-visual congruent walkable auralizations. An efficient method produces auralizations of acoustical landmarks using a two-dimensional ray-tracing algorithm and publicly available floor plans for a 128-channel wave-field synthesis system. Late reverberation parameters are calculated using additional volumetric data. Congruent visuals are produced using a web-based interface accessible via personal devices, which automatically formats for and transmits to the immersive display. Massive user-contributed online databases are harnessed through application programming interfaces, such as those offered by the Google Maps Platform, to provide near-instant access to innumerable locations. The approach allows the rapid sonic recreation of historical concert venues with adequate sound sources. Listeners can walk through these recreations over an extended user area (12 m × 10 m).

Perceptual Displays Today & Tomorrow – Evangelism, Productization, and Evaluation

Evolution of displays in various industries such as consumer electronics, aerospace, automotive, gaming and digital signage has been a key factor to engage and satisfy consumers. In this talk, I focus on Automotive market as one of the fastest growing display markets, but most discussed matters are applicable to other display verticals. Perceptual and Immersive display user experience are attracting more manufacturers these days to build products matching the end user’s expectation and needs while interacting with displays such as: perfect visibility in all conditions, personalization, less eye fatigue, seamless interaction - in one word “mimic real world” experience. Hardware and software advancements are critical but not enough to fulfil today’s knowledgeable consumer demands. The User Experience has become one of the highest priorities leading companies to develop dedicated user experience or UX teams to study the need of the end user. Multi-disciplinary teams of experts are built to tackle the UX optimization challenge._x005F_x000D_ _x005F_x000D_ In this talk I will share our experience evangelizing the benefits of Perceptual displays, challenges of such evangelism, productization of it and most importantly measuring the performance of perceptual displays in a comparable way to traditional displays.

Urban Landscape Information Construction and Visual Communication Design Based on Digital Image Matrix Reconstruction

This paper adopts the matrix reconstruction method to analyze the digital image of urban landscape information in-depth and uses it to study and design the visual communication of its construction. Based on the special structure and properties of Toeplitz matrices, a stepwise structured Augmented Lagrange Multiplier (SALM) algorithm for Toeplitz matrix filling is proposed by introducing structured operators. The main idea of the algorithm is to structure the iteration matrix at each step, that is, to reassign the elements on each diagonal of the matrix by the operator. In this way, the approximation matrix always maintains the Toeplitz structure during the iterative process, and the fast singular value decomposition of the Toeplitz. A matrix can be exploited, thus saving time. The convergence theory of the new algorithm is further discussed. A lightweight progressive feature fusion module is designed to improve network learning efficiency, which consists of two components: progressive local connectivity and feature attention. Specifically, progressive local connectivity extracts multilevel features for fusion by layer-by-layer separation and local splicing. Feature attention evaluates the importance of features using both channel and spatial dual attention modules. The Oculus Rift virtual reality device system is used, and the OSG graphics rendering engine is used as the basis for the scene data transfer of the overall 3D cityscape towards the virtual reality headset display device system. With the support of the Oculus SDK, the secondary rendering of the overall 3D cityscape in the immersive virtual reality module is carried out, and the corresponding OSG camera browsing interface is constructed to realize the two-eye immersive virtual display of the 3D cityscape in a virtual reality headset display device, assisting in connecting the virtual cityscape to reality.

A Design Method of Virtual Folk Museum Roaming System Based on Visual Interaction Technology

The effective display of museums can greatly improve the attention of museums. The traditional museum display methods are old and old-fashioned. Aiming at the lack of vivid and immersive display in folklore museums, based on augmented reality technology and artificial intelligence, a virtual folklore museum roaming system based on visual interaction technology is proposed for 3D visualization of cultural heritage. The method applies the 6D pose estimation algorithm of objects to the display of museum objects. 6D pose estimation of objects refers to detecting objects present in an image and estimating their 3D position and orientation relative to the observer. In the field of augmented reality, 6D pose estimation is used to measure the pose of objects in the real environment and add virtual objects to it with the correct pose, thereby realizing augmented reality applications on the mobile side. The experimental results show that the 6D pose estimation method proposed in this paper has an accuracy rate of 93.3% and a running speed of 30 frames per second, which meets the needs of augmented reality applications and can be used in augmented reality museum display systems to provide them with vivid interactive content. Breaking through the time and space limitations of cultural heritage objects, it brings an immersive visiting experience to the audience.

Interactive Cultural Communication Effect in VR Space of Intelligent Mobile Communication Network

Virtual reality technology can provide more display techniques to realize the interactive design of multiexhibition area of red humanities and natural resources. First of all, through the three-dimensional interactive display of cultural relics and scene reproduction of historical events, visitors can enjoy the unrestrained and immersive appreciation, make up for people’s lack of understanding of history, and let tourists personally feel the difficulties and hardships in the development of Chinese revolutionary history. Second, the digital method also provides for the pavilion and the scenic spot design personnel more creative method; dynamic scene reappearance interaction design based on virtual reality technology to promote education, protection of cultural relic collection and utilization, academic research, and the industrial development and multidimensional public welfare publicity results is blended in among them, through the panoramic, immersive, and interactive display technique, such as multilevel multiway spread red brigade culture. Virtual reality constantly imitates the real-life environment, and the application of perception sensing equipment has perception. This new technology requires us to continue to explore and research it. Virtual reality technology is a very cutting-edge subject and research field, which is very challenging. The communication form of culture has also been constantly evolving. Virtual reality technology (VR) has changed the way of traditional culture appreciation through the transmission of visual information, bringing people a new aesthetic enjoyment. Mainstream media is used to mobilize the red cultural heritage protection and scholars at home and abroad, to explore the application of virtual reality in the red culture tourist attraction planning, and to explore virtual reality applications in red open spot virtual roaming: exploration of virtual reality in the application of historical and cultural sites to red tourism management and exploration of virtual reality in the brand historical and cultural sites in the application of the travel marketing, to build an intelligent mobile communication network communication platform and broaden the new path of inheritance and protection. On the basis of studying the communication effect of traditional culture, this paper studies the possible communication effect of symphony orchestra under the new technology format under the framework of communication studies. It also analyzes and explores various factors such as the practical value, difficulty, civilized prestige, adaptability, and resistance to adversity of the culture in cultural communication and compares the methods of cultural communication. By developing a virtual instrument placed in the sphere of virtual reality space, the traditional symphony orchestra can be transformed into an interactive culture communication in virtual reality space. In this communication design, users can define their own instruments to play interactive culture communication, change the position of the instrument on the sphere interactively, and even change the sound effect of the symphony interactively by rotating or adjusting the size of the sphere, so as to achieve the best communication effect of the symphony.

Effects on Co-Presence of a Virtual Human: A Comparison of Display and Interaction Types

Recently, artificial intelligence (AI)-enabled virtual humans have been widely used in various fields in our everyday lives, such as for museum exhibitions and as information guides. Given the continued technological innovations in extended reality (XR), immersive display devices and interaction methods are evolving to provide a feeling of togetherness with a virtual human, termed co-presence. With regard to such technical developments, one main concern is how to improve the experience through the sense of co-presence as felt by participants. However, virtual human systems still have limited guidelines on effective methods, and there is a lack of research on how to visualize and interact with virtual humans. In this paper, we report a novel method to support a strong sense of co-presence with a virtual human, and we investigated the effects on co-presence with a comparison of display and interaction types. We conducted the experiment according to a specified scenario between the participant and the virtual human, and our experimental study showed that subjects who participated in an immersive 3D display with non-verbal interaction felt the greatest co-presence. Our results are expected to provide guidelines on how to focus on constructing AI-based interactive virtual humans.

The Effects of Interactivity on Learners’ Experience in a Visually Immersive Display Context

This research reports empirical evidence of the pedagogical benefits of using two interactive types of immersive spherical dome screens (large and small). Results are drawn from an experiment measuring the emotional, cognitive, perceptual/attitudinal states, and learning outcomes of 102 participants divided into control/experimental groups and group/individual pedagogical contexts. Specifically, this paper aimed to investigate the extent to which using interactivity in immersive technologies has positive learning outcomes and provide a positive learning experience. Building upon the INTERACT model, our results show that, in a smaller dome, the learner’s cognitive state, perception and attitudes play a mediating role regarding the impact of interactivity on learning outcomes. However, we did not observe a difference between the larger and smaller dome as for the learning outcomes. These results bring further insight into optimal educational practices in an immersive context and are relevant for pedagogical designers and immersive technologies manufacturers.

Immersive audio-visual scene reproduction using semantic scene reconstruction from 360 cameras

As personalised immersive display systems have been intensely explored in virtual reality (VR), plausible 3D audio corresponding to the visual content is required to provide more realistic experiences to users. It is well known that spatial audio synchronised with visual information improves a sense of immersion but limited research progress has been achieved in immersive audio-visual content production and reproduction. In this paper, we propose an end-to-end pipeline to simultaneously reconstruct 3D geometry and acoustic properties of the environment from a pair of omnidirectional panoramic images. A semantic scene reconstruction and completion method using a deep convolutional neural network is proposed to estimate the complete semantic scene geometry in order to adapt spatial audio reproduction to the scene. Experiments provide objective and subjective evaluations of the proposed pipeline for plausible audio-visual VR reproduction of real scenes.

Delta.AR: An augmented reality-based visualization platform for 3D genome

Delta.AR: An augmented reality-based visualization platform for 3D genome

Application possibilities of virtual reality in failure analysis of conveyor belts

Failure analysis of conveyor belts is realised using a wide range of the scientific methods. Visualisation of damage plays a very important role in performing of the given failure analysis. Development of the exponential technologies offer a possibility to apply the damage visualisation for analysis of failures in the area of rubber-textile conveyor belts. Virtual reality enables to create an immersive display, what can be very beneficial in the process of failure analysis. The basic idea of this article is to verify whether application of the virtual reality is suitable for purposes of the failure analysis concerning the rubber-textile conveyor belts. Application possibility of the virtual reality will be investigated and presented on example of the failure analysis for a test specimen selected from the damaged conveyor belt. The main task of this analysis is to discover a damage in the conveyor belt specimen, to evaluate this damage and to analyse the damage status of the individual layers creating internal structure of the conveyor belt. At the same time, there is presented in this article a methodology how to realise the failure analysis using a support of the virtual reality. Till now, there is not known any scientific-research work focused on the failure analysis of the conveyor belts, which would be based on the virtual reality. This article emphasizes application possibilities of the virtual reality within the given research area.

Design and Research of Interactive Animation of Immersive Space Scene Based on Computer Vision Technology

With the development of computational simulation technology, the need for stable and immersive display effects of space scene animation in the field of life experience and visual art has gradually increased. In this case, the requirements for immersive characteristics of space scene animation have also been strengthened. The existing 3D space scene animation has a limited degree of stereoscopic display model and data visualization. However, the current space scene interactive animation adopts a plane layout as a whole, and the size of the view interface is generally fixed, including the size of virtual elements. However, the immersion in this paper interactive animation of spatial scene can effectively solve the problems of incomplete display of 3D effects and unstable view simulation in 3D effects. This paper takes immersive space scene animation as the research object and studies the 3D and characteristics of space scene animation based on computer parallel computing in different immersive space scenes and different virtual space technologies, as well as the animation effects of different scene transformations and art forms. The result of research shows that, with the continuous increase of the degree of virtuality within a certain range, the immersive effect of spatial scene interactive animation gradually becomes better. When the color of the space scene animation is below 15, the virtual immersive effect changes less. When the space scene is in the range of 15–20, the space scene will make people feel the atmosphere of a beautiful and mysterious illusion.

Dual OS Smart Mirror Using Raspberry Pi (Allure Mirror)

This paper describes the designing and implementation of a dual operating system wall mirror, called “Allure Mirror”. It is a device that can act as a mirror as well as a dual -OS immersive display which can display time, date, weather, news, can play video (via. YouTube, Prime Video, Netflix, etc.), can do video calls, use social media, check mails, can help in fitness and many more. Voice instructions can be used to communicate with it. The Allure Mirror includes features such as real-time data and information alerts, voice orders, content viewing via LCD display, microphone, and webcam, and more. Voice instructions can be used to communicate with the Allure mirror.