Virtual Content Research Articles

An augmented reality approach for communicating intangible and architectural heritage through digital characters and scale models

Intangible cultural heritage (ICH) represents living cultural expressions and practices that are part of the heritage of a community, and their preservation and transmission are considered highly important. Various methods and tools have been applied so far for the digitization and dissemination of ICH content including a wide range of technologies. Mobile augmented reality is a promising solution along this path that enables the overlap of digital and real-world information in an engaging and efficient manner. Despite the widespread use of AR in cultural heritage, there are not many studies regarding the user experience, the learning outcomes, and the way in which users observe and interact with the virtual content. This paper presents a mobile augmented reality installation that re-enacts the stages of leather tanning process, adopting a novel approach that augments 3D content upon a physical scale model of an old tannery. This approach pursues to transmit the cultural value of traditional craftmanship to visitors of the building and associate its architectural elements to its history and use. A user evaluation was conducted aiming to measure the users’ engagement, learning, and experience using the installation. The encouraging results led to a follow-up study about the impact of the physical scale model on the experience. Two variations of the experience have been studied, one with a physical scale model and one with a digital-only version in a between-subject design. The results of the two studies provide evidence that the proposed approach generated a positive user experience and evident learning gain and was considered easy to use, highlighting its potential to be widely adopted in buildings with architectural value.

On the issue of preparing future teachers to broadcast family values to students in an information society

This article examines the potential and features of the formation at the university of future teachers of professional readiness to develop and transmit family values to students. The relevance of the topic is determined by the crisis state of the modern family and the need to broadcast the values of the traditional family to the younger generation. The author understands the peculiarities of this process as the originality of the modern generation of youth, the specifics of the activities of teachers of pedagogical training profiles at the university, as well as a set of targeted, substantive and procedural features that are modified in the situation of transformation of the axiosphere of a modern citizen. The set of personality-oriented, socially-oriented, civic-oriented, artistically-oriented, reflexively-oriented and professionally-oriented groups of family values highlighted by the author is purposefully formed in the selected educational conditions of family communication, leisure activities, public associations, as well as organizations of general, additional and vocational education. Examples of family education on educational platforms with virtual content, examples of the use of hackathons in university education, a professional skills competition, and STEAM design are considered. Generalized features of the formation and transmission of family values to students by future teachers in university education (adaptability to a specific place, time, amount of study load, hybridity, continuity, the ability to return to some material, personalized pace, volume, style and support of education with the possibilities of self-government and its adjustments) arouse interest among modern students and increase their professional motivation.

Vox-Surf: Voxel-Based Implicit Surface Representation.

Virtual content creation and interaction play an important role in modern 3D applications. Recovering detailed 3D models from real scenes can significantly expand the scope of its applications and has been studied for decades in the computer vision and computer graphics community. In this work, we propose Vox-Surf, a voxel-based implicit surface representation. Our Vox-Surf divides the space into finite sparse voxels, where each voxel is a basic geometry unit that stores geometry and appearance information on its corner vertices. Due to the sparsity inherited from the voxel representation, Vox-Surf is suitable for almost any scene and can be easily trained end-to-end from multiple view images. We utilize a progressive training process to gradually cull out empty voxels and keep only valid voxels for further optimization, which greatly reduces the number of sample points and improves inference speed. Experiments show that our Vox-Surf representation can learn fine surface details and accurate colors with less memory and faster rendering than previous methods. The resulting fine voxels can also be considered as the bounding volumes for collision detection, which is useful in 3D interactions. We also show the potential application of Vox-Surf in scene editing and augmented reality. The source code is publicly available at https://github.com/zju3dv/Vox-Surf.

A real-time wearable AR system for egocentric vision on the edge

Real-time performance is critical for Augmented Reality (AR) systems as it directly affects responsiveness and enables the timely rendering of virtual content superimposed on real scenes. In this context, we present the DARLENE wearable AR system, analysing its specifications, overall architecture and core algorithmic components. DARLENE comprises AR glasses and a wearable computing node responsible for several time-critical computation tasks. These include computer vision modules developed for the real-time analysis of dynamic scenes supporting functionalities for instance segmentation, tracking and pose estimation. To meet real-time requirements in limited resources, concrete algorithmic adaptations and design choices are introduced. The proposed system further supports real-time video streaming and interconnection with external IoT nodes. To improve user experience, a novel approach is proposed for the adaptive rendering of AR content by considering the user’s stress level, the context of use and the environmental conditions for adjusting the level of presented information towards enhancing their situational awareness. Through extensive experiments, we evaluate the performance of individual components and end-to-end pipelines. As the proposed system targets time-critical security applications where it can be used to enhance police officers’ situational awareness, further experimental results involving end users are reported with respect to overall user experience, workload and evaluation of situational awareness.

Hunting for pearls in the shallow water: designing and building spatial immateriality in the aquatic theatre La Perle

This essay discusses the working process behind Franco Dragone’s aquatic theatre La Perle (2017) in Dubai. The aim of the case study is to provide unique insight into the development and implementation of an innovative architecture typology of a theatre building conceived as a ‘living thing’, a spatial system able to adapt and change according to the demands of dramaturgy. Remembering imaginative, but never built, proposals for theatre buildings from the beginning of the twentieth century, and their objectives regarding more flexibility of a performance space, the study explains what it takes to actually realise a ‘fluid’ spatial system. At the same time, it is an attempt to broaden the understanding of scenography by discussing it in relation to its materiality, technical requirements, and the virtual content employed in the La Perle show. Using the lens of posthumanism, the analysis includes discussion on the culture/nature divide in the realm of contemporary spectacle, concentrating on Karen Barad’s notion of non-human matter and its role within an aquatic theatre, adaptable and ever-changing. It explains an approach to treating space as a scenographic body, and the contribution of architectural design proceedings to the conception and production of new kinds of theatres.

An Enhancement of Outdoor Location-Based Augmented Reality Anchor Precision through VSLAM and Google Street View.

Outdoor Location-Based Augmented Reality (LAR) applications require precise positioning for seamless integrations of virtual content into immersive experiences. However, common solutions in outdoor LAR applications rely on traditional smartphone sensor fusion methods, such as the Global Positioning System (GPS) and compasses, which often lack the accuracy needed for precise AR content alignments. In this paper, we introduce an innovative approach to enhance LAR anchor precision in outdoor environments. We leveraged Visual Simultaneous Localization and Mapping (VSLAM) technology, in combination with innovative cloud-based methodologies, and harnessed the extensive visual reference database of Google Street View (GSV), to address the accuracy limitation problems. For the evaluation, 10 Point of Interest (POI) locations were used as anchor point coordinates in the experiments. We compared the accuracies between our approach and the common sensor fusion LAR solution comprehensively involving accuracy benchmarking and running load performance testing. The results demonstrate substantial enhancements in overall positioning accuracies compared to conventional GPS-based approaches for aligning AR anchor content in the real world.

Designing a 3D gestural interface to support user interaction with time-oriented data as immersive 3D radar charts

The design of intuitive three-dimensional user interfaces is vital for interaction in virtual reality, allowing to effectively close the loop between a human user and the virtual environment. The utilization of 3D gestural input allows for useful hand interaction with virtual content by directly grasping visible objects, or through invisible gestural commands that are associated with corresponding features in the immersive 3D space. The design of such interfaces remains complex and challenging. In this article, we present a design approach for a three-dimensional user interface using 3D gestural input with the aim to facilitate user interaction within the context of Immersive Analytics. Based on a scenario of exploring time-oriented data in immersive virtual reality using 3D Radar Charts, we implemented a rich set of features that is closely aligned with relevant 3D interaction techniques, data analysis tasks, and aspects of hand posture comfort. We conducted an empirical evaluation (n=12)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$(n=12)$$\\end{document}, featuring a series of representative tasks to evaluate the developed user interface design prototype. The results, based on questionnaires, observations, and interviews, indicate good usability and an engaging user experience. We are able to reflect on the implemented hand-based grasping and gestural command techniques, identifying aspects for improvement in regard to hand detection and precision as well as emphasizing a prototype’s ability to infer user intent for better prevention of unintentional gestures.

A scope of presence-related feelings in AR studies

The idiosyncrasies of augmented reality bring us advantages, as shown in literature, and a great assortment of options. On the other hand, due to its relationship with the real surroundings, it may be a challenge to deal with when evaluating these systems, especially if the intention is to understand feelings of presence-like: There are a lot of variables in the equation. This study aims to analyse a state of the art of AR evaluations that conducted presence-related feelings and discusses limitations and remarks for further research. The current research is able to state that questionnaires are the most used tool to ascertain presence-like feelings, and that mobile devices have been the preferred device to implement AR applications. The studies are fairly divided between 1) analysing a single scenario to ascertain the variable at study, and 2) creating two or more scenarios to make a comparison. When comparing two or more scenarios, a between-subjects design is preferred among researchers. Additionally, it has been identified as of paramount importance the need to study and objectively measuring the ratio between the virtual content and the real scenario in the experience, as well as to deeply research the interaction between the real scenario and the virtual elements. The importance of the types of interaction in AR applications is also highlighted.

Optical Rules to Mitigate the Parallax-Related Registration Error in See-Through Head-Mounted Displays for the Guidance of Manual Tasks

Head-mounted displays (HMDs) are hands-free devices particularly useful for guiding near-field tasks such as manual surgical procedures. See-through HMDs do not significantly alter the user’s direct view of the world, but the optical merging of real and virtual information can hinder their coherent and simultaneous perception. In particular, the coherence between the real and virtual content is affected by a viewpoint parallax-related misalignment, which is due to the inaccessibility of the user-perceived reality through the semi-transparent optical combiner of the OST Optical See-Through (OST) display. Recent works demonstrated that a proper selection of the collimation optics of the HMD significantly mitigates the parallax-related registration error without the need for any eye-tracking cameras and/or for any error-prone alignment-based display calibration procedures. These solutions are either based on HMDs that projects the virtual imaging plane directly at arm’s distance, or they require the integration on the HMD of additional lenses to optically move the image of the observed scene to the virtual projection plane of the HMD. This paper describes and evaluates the pros and cons of both the suggested solutions by providing an analytical estimation of the residual registration error achieved with both solutions and discussing the perceptual issues generated by the simultaneous focalization of real and virtual information.

MOUNT: Learning 6DoF Motion Prediction Based on Uncertainty Estimation for Delayed AR Rendering.

The delay of rendering on AR devices requires prediction of head motion using sensor data acquired tens of even one hundred milliseconds ago to avoid misalignment between the virtual content and the physical world, where the misalignment will lead to a sense of time latency and dizziness for users. To solve the problem, we propose a method for the 6DoF motion prediction to compensate for the time latency. Compared with traditional hand-crafted methods, our method is based on deep learning, which has better motion prediction ability to deal with complex human motion. In particular, we propose a MOtion UNcerTainty encode decode network (MOUNT) that estimates the uncertainty of input data and predicts the uncertainty of output motion to improve the prediction accuracy and smoothness. Experiments on the EuRoC and our collected dataset demonstrate that our method significantly outperforms the traditional method and greatly improves AR visual effects.

Prosthodontic Research in the Digital Era: Current Trends and Future Prospects

Prosthodontic Research in the Digital Era: Current Trends and Future Prospects

Workspace Guardian: Investigating Awareness of Personal Workspace Between Co-Located Augmented Reality Users.

As augmented reality (AR) systems proliferate and the technology gets smaller and less intrusive, we imagine a future where many AR users will interact in the same physical locations (e.g., in shared work places and public spaces). While previous research has explored AR collaboration in these spaces, our focus is on co-located but independent work. In this paper, we explore co-located AR user behavior and investigate techniques for promoting awareness of personal workspace boundaries. Specifically, we compare three techniques: showing all virtual content, visualizing bounding box outlines of content, and a self-defined workspace boundary. The findings suggest that a self-defined boundary led to significantly more personal workspace encroachments.

100-Phones: A Large VI-SLAM Dataset for Augmented Reality Towards Mass Deployment on Mobile Phones.

Visual-inertial SLAM (VI-SLAM) is a key technology for Augmented Reality (AR), which allows the AR device to recover its 6-DoF motion in real-time in order to render the virtual content with the corresponding pose. Nowadays, smartphones are still the mainstream devices for ordinary users to experience AR. However the current VI-SLAM methods, although performing well on high-end phones, still face robustness challenges when deployed on a larger stock of mid- and low-end phones. Existing VI-SLAM datasets use either very ideal sensors or only a limited number of devices for data collection, which cannot reflect the capability gaps that VI-SLAM methods need to solve when deployed on a large variety of phone models. This work proposes 100-Phones. the first VI-SLAM dataset covering a wide range of mainstream phones in the market. The dataset consists of 350 sequences collected by 100 different models of phones. Through analysis and experiments on the collected data, we conclude that the quality of visual-inertial data vary greatly among the mainstream phones, and the current open source VI-SLAM methods still have serious robustness issues when it comes to mass deployment on mobile phones. We release the dataset to facilitate the robustness improvement of VI-SLAM and to promote the mass popularization of AR. Project page: https://github.com/zju3dv/100-Phones.

AMFL: Resource-Efficient Adaptive Metaverse-Based Federated Learning for the Human-Centric Augmented Reality Applications.

The emergence of 5G technology has enabled the development of Metaverse applications that provide users with immersive experiences through augmented reality (AR) devices, and the integration of federated learning (FL) with the Metaverse AR (MAR) systems can enable many edge intelligence services in 5G. However, the presence of nonindependent and identically distributed (Non-IID) data across all AR users' devices, coupled with limited edge communication resources, makes it challenging to achieve human-centric Metaverse-related applications such as target detection or image classification that combine virtual content with real-world. To address these challenges, we propose a novel adaptive resource-efficient Metaverse-based FL (AMFL) algorithm for AR applications that mitigates the negative effect of Non-IID data and reduces resource costs as well as improves the quality of experience (QoE). We first analyze the impact of wireless communication factors such as CPU frequency, bandwidth, and transmission power on FL training performance by a toy example in the MAR systems. Based on this analysis, furthermore, we establish a Non-IID degree, model accuracy, and resource consumption-related QoE maximization problem under given resource budgets, which is a stochastic optimization problem with strongly coupled variables, including bandwidth, CPU frequency, and transmission power. Guided by the theoretical analysis, to solve this issue, AMFL employs a deep reinforcement learning (DRL)-based method to adaptively allocate resources. Numerical results demonstrate that AMFL can significantly improve the QoE by up to 30.28 % , and reduce communication round and energy costs by up to 81.08 % and 72.20 % , respectively, even under the worst Non-IID case, compared to benchmarks.

Análise da Oferta de Empreendimentos Turísticos Associados ao Veganismo

Veganism is a social movement that seeks to end animal exploitation and embraces individual and collective practices to achieve this purpose in the social, environmental, economic and political spheres. Such practices mainly affect the food market, but extend to other economic sectors, such as the tourist market. Therefore, the present study aims to evaluate the relevant adaptations to the vegan niche inserted in this market. The methodological course comprised an exploratory study with a qualitative approach using virtual content analysis and semi-structured interview. Regarding the virtual content analysis, reviews mentioning the terms “vegano/a” on TripAdvisor were searched to find travel agencies and accommodation facilities specialized in serving vegans in the states of São Paulo and Rio de Janeiro. Based on the result of this tourist mapping an after the analysis of their social medias, certain enterprises had their representatives invited to participate in interviews about the services provided to the vegan public. The content analysis of the interviews was carried out from the following categories: gastronomy; sustainability; experiences; hospitality. Among the results of the survey, there was an increase in the offer of tourist products suitable for the vegan public, and how enterprises have been seeking practices that are increasingly oriented to socio-environmental issues, to attract not only vegans, but conscious consumers. This research contributed to the understanding of how veganism has been transforming the tourist market and to the consolidation of a synthesis of management indicators for enterprises that wish to participate in the vegan travel niche.

The 4C framework: Towards a holistic understanding of consumer engagement with augmented reality

Augmented Reality (AR) is an emerging concept that impacts many disciplines, such as business, marketing, tourism, gaming, human–computer interaction, and manufacturing. Surprisingly, many scholarly and practical discussions overlook the fundamental primary factors that distinguish AR from other concepts, namely, that it involves a computing device that integrates virtual content into a consumer's perception of the real world in a specific context. The current article addresses this gap in the literature by proposing the 4C framework (based on the 4Cs: consumer, content, context, and computing device; pronounced: foresee) that highlights the importance of, and interplay among these four factors. Building on configurational theory, the framework calls for the systematic identification of additional AR-relevant factors across the 4Cs. Scholars can use this framework to systematically identify research gaps and variables of interest. Practitioners across various disciplines can employ the framework to systematically assess, communicate, and develop AR use cases.

Blind people can actively manipulate virtual objects with a novel tactile device

Frequently in rehabilitation, visually impaired persons are passive agents of exercises with fixed environmental constraints. In fact, a printed tactile map, i.e. a particular picture with a specific spatial arrangement, can usually not be edited. Interaction with map content, instead, facilitates the learning of spatial skills because it exploits mental imagery, manipulation and strategic planning simultaneously. However, it has rarely been applied to maps, mainly because of technological limitations. This study aims to understand if visually impaired people can autonomously build objects that are completely virtual. Specifically, we investigated if a group of twelve blind persons, with a wide age range, could exploit mental imagery to interact with virtual content and actively manipulate it by means of a haptic device. The device is mouse-shaped and designed to jointly perceive, with one finger only, local tactile height and inclination cues of arbitrary scalar fields. Spatial information can be mentally constructed by integrating local tactile cues, given by the device, with global proprioceptive cues, given by hand and arm motion. The experiment consisted of a bi-manual task, in which one hand explored some basic virtual objects and the other hand acted on a keyboard to change the position of one object in real-time. The goal was to merge basic objects into more complex objects, like a puzzle. The experiment spanned different resolutions of the tactile information. We measured task accuracy, efficiency, usability and execution time. The average accuracy in solving the puzzle was 90.5%. Importantly, accuracy was linearly predicted by efficiency, measured as the number of moves needed to solve the task. Subjective parameters linked to usability and spatial resolutions did not predict accuracy; gender modulated the execution time, with men being faster than women. Overall, we show that building purely virtual tactile objects is possible in absence of vision and that the process is measurable and achievable in partial autonomy. Introducing virtual tactile graphics in rehabilitation protocols could facilitate the stimulation of mental imagery, a basic element for the ability to orient in space. The behavioural variable introduced in the current study can be calculated after each trial and therefore could be used to automatically measure and tailor protocols to specific user needs. In perspective, our experimental setup can inspire remote rehabilitation scenarios for visually impaired people.

Accommodation-capable holographic waveguide head-up display with extended field of view

Augmented reality (AR) displays are enhancing user experiences by offering immersive three- dimensional (3D) content, with head-up display (HUD) being a prime application for driving safety. To enable AR-HUDs to provide sufficient information to driver, it is essential to ensure a wide field of view (FOV). Traditional methods like the magnifier principle have limitations stemming from optical aberrations and bulky form factor. Recently, optical waveguide has been used as pupil expander, which has attracted increasing attention as an optical element for compact form factor and exit-pupil expansion. However, waveguide display as exit-pupil expander only offer 2D images with infinite depth, causing mismatch issue between virtual content and real scenes. In this paper, we propose a lensless holographic waveguide display consisting of a laser light source, a spatial light modulator (SLM), a waveguide to generate accommodation-capable images, effectively extending FOV. The key distinction of this research lies in the optical design of the waveguide, which defines multiple shifted copies of the modulated wavefield as replicated virtual SLMs. A formalized algorithm is devised based on a bold assumption that numerous virtual SLMs replicated by a waveguide are considered as a single ultra-high resolution SLM. The effectiveness of the algorithm has been demonstrated through extensive validation through numerical observation simulations and optical experiments. Notably, optical experiments convincingly demonstrate the system’s ability to produce accommodation-capable true 3D content, with a four-fold extension of FOV compared to a single SLM at an observation distance of 150 mm. Additional optical experiments highlight the successful integration of AR technology, an important component for automotive HUD.

Why Purchase Premium in Freemium Online Games: The Self Determination Theory Perspective

After the rise of the freemium business model, the video games industry emerged as a popular source of entertainment in Malaysia. In this modern free game model, virtual content trading is considered a great source of revenue generation for game developers and publishers. Millions of players around the world participate in this activity and purchase virtual items. In this study, we focus on exploring the motives of Malaysian youth for purchasing these in-game virtual products. By employing purposive sampling, qualitative data were collected by means of conducting in-depth interviews with game players who purchase virtual products. This study explored seven important factors: the desire for uniqueness, the desire for dominance, the desire for skillfulness, social interaction, the desire for freedom, social affiliation, and enjoyment, which strongly influence Malaysian players’ behavior toward in-game content consumption during video game playing. Gamers and game developers can benefit from the findings of the present study to customize the in-game virtual product design strategies as per the expectations of respondents.
 Keywords: free-to-play, video games, freemium, virtual purchasing, purchase intention

Training with, about, for Metaverse: A Mixed Methods Research on Training Pre-Service Teachers as Metaverse-Certified Practitioners

This mixed-method research aims to examine transformative affordances of the metaverse for pre-service teachers and investigate support mechanisms for technology-enhanced teacher training. Specifically, 40 pre-service teachers were involved in training <i>with, about, for</i> metaverse (TM) to convert high-stakes TOEIC test items into virtual content. Upon completion of the TM, participants were eligible to apply for nationally accredited private metaverse certificates to mark them as certified practitioners. To capture their multifaceted TM experience, this research employed surveys based on UTAUT (Unified Theory of Acceptance and Use of Technology) 2 constructs, as well as reflective journals and advanced analytics. The collected data were analyzed using quantitative techniques (<i>t</i>-tests, multiple regression analysis, and structural equation modeling) and qualitative methods (sentiment and content analysis) to offer a more comprehensive understanding of the technological training for pre-service teachers. The findings revealed that the overall reception of the TM experience among pre-service teachers was highly positive. Notable improvements were observed in facilitating conditions, effort expectancy, and hedonic motivation. Textmining results also revealed how the TM framework transformed pre-service teachers’ perspectives on teaching, testing, and learning. Based on the findings, the study contributes to the ongoing dialogue on effective technology integration in education.