Designs and Interactions for Near-Field Augmented Reality: A Scoping Review

Creating pedagogically sound, interactive Augmented Reality (AR) experiences supporting situated and experiential learning remains a challenge to teachers without programming skills. To integrate AR in the everyday classroom, teachers need to be capable of designing their own immersive experiences for their students, which is why an analysis of existing authoring toolkits is necessary to identify suitable tools for educational application development and future research directions in terms of educational AR. We identified “easy access”, “GUI-based design”, and “interactive contents” as needs of teachers for designing AR content for the classroom. Based on these needs, we conducted a literature review of 835 documents. Of 80 relevant articles, we included 43 peer-reviewed articles from ACM Digital Library, DBLP, IEEExplore, Scopus, Web of Science, Google Scholar, and miscellaneous other sources in our analysis. We identified 69 different AR authoring toolkits and classified these with regard to their accessibility, their degree of required programming knowledge, and their interactivity. The results show a divergent research landscape with a lack of empirical evaluation. Of 26 openly accessible toolkits, we identified five toolkits addressing the defined needs of teachers for designing interactive AR experiences for the classroom without requiring extensive programming knowledge. We conclude that there are only few tools for the straightforward design of educational AR experiences addressing the needs of teachers and suggest using research-informed and evidence-based criteria for developing AR authoring toolkits for education.

Reality check

Aim/Purpose: Designing augmented reality (AR) experiences for education, health or entertainment involves multidisciplinary teams making design decisions across several areas. The goal of this paper is to present a classification schema that describes the design choices when constructing an AR interactive experience. Background: Existing extended reality schema often focuses on single dimensions of an AR experience, with limited attention to design choices. These schemata, combined with an analysis of a diverse range of AR applications, form the basis for the schema synthesized in this paper. Methodology: An extensive literature review and scoring of existing classifications were completed to enable a definition of seven design dimensions. To validate the design dimensions, the literature was mapped to the seven-design choice to represent opportunities when designing AR iterative experiences. Contribution: The classification scheme of seven dimensions can be applied to communicating design considerations and alternative design scenarios where teams of domain specialists need to collaborate to build AR experiences for a defined purpose. Findings: The dimensions of nature of reality, location (setting), feedback, objects, concepts explored, participant presence and interactive agency, and style describe features common to most AR experiences. Classification within each dimension facilitates ideation for novel experiences and proximity to neighbours recommends feasible implementation strategies. Recommendations for Practitioners: To support professionals, this paper presents a comprehensive classification schema and design rationale for AR. When designing an AR experience, the schema serves as a design template and is intended to ensure comprehensive discussion and decision making across the spectrum of design choices. Recommendations for Researchers: The classification schema presents a standardized and complete framework for the review of literature and AR applications that other researchers will benefit from to more readily identify relevant related work. Impact on Society: The potential of AR has not been fully realized. The classification scheme presented in this paper provides opportunities to deliberately design and evaluate novel forms of AR experience. Future Research: The classification schema can be extended to include explicit support for the design of virtual and extended reality applications.

Aim/Purpose: Designing augmented reality (AR) experiences for education, health or entertainment involves multidisciplinary teams making design decisions across several areas. The goal of this paper is to present a classification schema that describes the design choices when constructing an AR interactive experience. Background: Existing extended reality schema often focuses on single dimensions of an AR experience, with limited attention to design choices. These schemata, combined with an analysis of a diverse range of AR applications, form the basis for the schema synthesized in this paper. Methodology: An extensive literature review and scoring of existing classifications were completed to enable a definition of seven design dimensions. To validate the design dimensions, the literature was mapped to the seven-design choice to represent opportunities when designing AR iterative experiences. Contribution: The classification scheme of seven dimensions can be applied to communicating design considerations and alternative design scenarios where teams of domain specialists need to collaborate to build AR experiences for a defined purpose. Findings: The dimensions of nature of reality, location (setting), feedback, objects, concepts explored, participant presence and interactive agency, and style describe features common to most AR experiences. Classification within each dimension facilitates ideation for novel experiences and proximity to neighbours recommends feasible implementation strategies. Recommendations for Practitioners: To support professionals, this paper presents a comprehensive classification schema and design rationale for AR. When designing an AR experience, the schema serves as a design template and is intended to ensure comprehensive discussion and decision making across the spectrum of design choices. Recommendations for Researchers: The classification schema presents a standardized and complete framework for the review of literature and AR applications that other researchers will benefit from to more readily identify relevant related work. Impact on Society: The potential of AR has not been fully realized. The classification scheme presented in this paper provides opportunities to deliberately design and evaluate novel forms of AR experience. Future Research: The classification schema can be extended to include explicit support for the design of virtual and extended reality applications.

With Generation Z becoming the primary group of nursing learners and the increasing shortage of nursing education resources, augmented reality and mixed reality based on head-mounted displays are being used more and more in nursing education. However, the current application landscape and the proper usage of these concepts remain unclear. Therefore, this study aims to conduct a scoping review to explore the current applications of head-mounted display-based augmented reality and mixed reality in nursing education and to clarify the definitions and usage of the concepts of augmented reality and mixed reality, ultimately providing directions for future applications and research. Based on the five-stage framework and PRISMA-ScR guidelines, a comprehensive collection and summarization of evidence regarding the application of head-mounted display-based augmented reality and mixed reality in nursing education were conducted. The databases retrieved include CNKI, Wanfang Database, VIP, CBM, PubMed, Cochrane Library, Embase, CINAHL, Web of Science, Scopus, IEEE Xplore, ACM Digital Library, and Ei. The languages of the included literature are Chinese and English. The retrieval was up to October 2024. A total of 44 studies were included in this review, covering three types of head-mounted displays: immersive head-mounted displays, smart glasses, and smartphone-based head-mounted displays. The main application areas were skills training and knowledge acquisition. Most of the studies were feasibility studies, though they also included some efficacy studies and research on personal experiences. In addition, these studies often employed vague or inconsistent definitions of augmented reality and mixed reality. Despite various explorations in the application of head-mounted display-based augmented reality and mixed reality in nursing education, there is still room for improvement in the relevant theory and utilization of this technology. In the future, interventions should use the three dimensions (observation of reality, real - virtual interaction, and fidelity of virtuality) described in detail, rather than simply employing the concepts of augmented reality or mixed reality. Efforts should be concentrated on developing and implementing head-mounted displays combined with other technologies that boast enhanced performance and cost - effectiveness, and further validating their effectiveness.

Recently, Massive Open Online Courses have become sensational in the field of distance learning. There is a plethora of advantages being listed in learning through MOOCs but this pedagogy lacks in few areas when compared with traditional classes. One of those inabilities of MOOC is its support to prepare the students for laboratory-based courses. The authors of this study chose a MOOC course that teaches Digital Photography and created an Augmented Reality (AR) experience for a module that explains the different parts of a digital camera. The 2nd year Multimedia students of Vellore Institute of Technology have been asked to experience the MOOC video followed by the AR experience. Their feedbacks before and after the AR experience has been statistically tested and reported. The results revealed that the students feel more confident and concentrate more when the instructional video was given as an AR experience. This study suggests that AR integrated MOOC modules might help in training students better for practice-based courses.

Learning through augmented reality (AR) and virtual reality (VR) experiences has become a valuable approach in modern robotics education. This study evaluated this approach and investigated how 99 first-year industrial engineering students explored robot systems through such online experiences while staying at home. The objective was to examine learning in the AR/VR environment and evaluate its contribution to understanding the robot systems and to fostering integrative thinking. During the AR experiences that we developed using Vuforia Studio, the students learned about TurtleBot2 and RACECAR MN robots while disassembling and modifying their models and by obtaining information about their components. In the VR experience with the RacecarSim simulator, the students explored sensor-based robot navigation. Quizzes were used to assess understanding of robot systems, and a post-workshop questionnaire evaluated the workshop’s contribution to learning about the robots and to training integrative thinking skills. The data indicate that the students gained understanding of the robot systems, appreciated the contribution of the augmented and virtual reality apps, and widely used integrative thinking throughout the practice. Our study shows that AR apps and virtual simulators can be effectively used for experiential learning about robot systems in online courses. However, these experiences cannot replace practice with real robots.

Mixed reality (MR) interactions feature users interacting with a combination of virtual and physical components. Inspired by research investigating aspects associated with near-field interactions in augmented and virtual reality (AR & VR), we investigated how avatarization, the physicality of the interacting components, and the interaction technique used to manipulate a virtual object affected performance and perceptions of user experience in a mixed reality fundamentals of laparoscopic peg-transfer task wherein users had to transfer a virtual ring from one peg to another for a number of trials. We employed a 3 (Physicality of pegs) X 3 (Augmented Avatar Representation) X 2 (Interaction Technique) multi-factorial design, manipulating the physicality of the pegs as a between-subjects factor, the type of augmented self-avatar representation, and the type of interaction technique used for object-manipulation as within-subjects factors. Results indicated that users were significantly more accurate when the pegs were virtual rather than physical because of the increased salience of the task-relevant visual information. From an avatar perspective, providing users with a reach envelope-extending representation, though useful, was found to worsen performance, while co-located avatarization significantly improved performance. Choosing an interaction technique to manipulate objects depends on whether accuracy or efficiency is a priority. Finally, the relationship between the avatar representation and interaction technique dictates just how usable mixed reality interactions are deemed to be.

It gives us great pleasure to introduce this special issue of the Springer Virtual Reality Journal, dedicated to research in augmented reality (AR) in light of the growing interest in the field. Over 40 years ago, Ivan Sutherland developed the first AR application that allowed virtual images to be seamlessly overlaid onto the real world. Since then, much of the research in the field has been focused on the enabling technology for providing AR experiences, such as tracking, display devices, and software systems. However, in the call for papers, we asked for work with a focus on augmented reality experiences, such as new techniques for interacting with AR, methods for evaluating the quality of the experience, better tools for creating AR experiences for nonprogrammers and other related topics. We received more than thirty submissions, most of which were of a very high quality, and we are delighted to present the final selection of eleven papers. Each of these papers were reviewed by a least four reviewers and have been revised to address the reviewers feedback before appearing here. The papers can be grouped into three categories: Tracking and Projection Methods, Interaction Techniques, and Interface Design and Systems. In the Tracking group, (Lieberknecht et al. 2011) present methods for benchmarking template-based computer vision tracking algorithms for AR. Template-based tracking has been the most popular form of vision-based AR tracking for the past 10 years and their work gives valuable guidance on how to benchmark the performance of such systems. Uchiyama (2011) discusses how to do camera tracking of online learning of key points using a Locally Likely Arrangement Hashing technique which looks promising. Finally, (Nagase et al. 2011) shows how model-based optimal projector selection in multiprojection environment can be used to compensate for dynamic defocus and occlusion in projected imagery. The Interaction papers presented a range of different ways of interacting with AR content. Lee et al.’s work (2011) explores tangible interaction methods and, in particular, how two-handed tangible techniques can be used for arranging AR blocks. In their paper, they show how very natural block arranging methods with real blocks can be used for intuitive manipulation of virtual content. Iwai and Sato’s paper on the Limpid Desk (2011) presents an interesting AR projectionbased interface for seeing virtual imagery projected into real documents. They also use gesture-based interaction to enable touch sensing and browsing through the document stack. Finally, the (Ajanki et al. 2011) paper explores how to develop context-sensitive interaction techniques. Using position and gaze sensing, their system can automatically infer user’s interest in people and topics, and display AR content reflecting their interest and so support implicit rather than explicit interaction. The largest selection of papers is in the Interface Design and AR Systems group. Livingston’s paper (2011) presents general user interface guidelines for military AR applications and, in particular, for presenting outdoor AR content. Kim (2011) describes a related system which allows virtual video to be overlaid on aerial maps. The In-Place AR paper of (Hagbi et al. 2011) also shows an interesting map-based M. Billinghurst (&) HIT Lab NZ, University of Canterbury, Christchurch, New Zealand e-mail: mark.billinghurst@hitlabnz.org

PurposeThe current study explores two distinct augmented reality (AR) experiences (hedonic and utilitarian) to determine the drivers of satisfaction and word-of-mouth (WOM). Specifically, this research investigates how different AR characteristics (person–virtual environment (VE) interaction, novelty) impact reality congruence and immersion, which ultimately lead to satisfaction and WOM.Design/methodology/approachWe tested the proposed model using a US consumer sample (N = 401) by applying partial least squares structural equation modeling (PLS-SEM) to reveal differences across the AR experiences regarding the mediating effects of immersion and reality congruence. Furthermore, we administered PLS-SEM multigroup analysis to identify differences for hedonic and utilitarian AR experiences.FindingsThis study provides important insights into the use of AR to create value-driven consumer experiences and outcome behaviors. Interacting with the virtual environment (VE) enhances immersion for hedonic AR experiences while it leads to higher levels of reality congruence for utilitarian AR experiences. While established relationships such as novelty to immersion continue to be significant, new constructs such as person–VE interaction and reality congruence immerge as stronger influences.Originality/valueTo date, most studies have implemented apps such as IKEA Place and have considered value as one comprehensive construct. However, limited research has examined differences in hedonic and utilitarian AR experiences. The current findings enhance the understanding of utilizing value-driven consumer experiences within the AR context to achieve desirable outcome behaviors.

This paper presents an evaluation and comparison of interaction methods for the configuration and visualization of pervasive Augmented Reality (AR) experiences using two different platforms: desktop and mobile. AR experiences consist of the enhancement of real-world environments by superimposing additional layers of information, real-time interaction, and accurate 3D registration of virtual and real objects. Pervasive AR extends this concept through experiences that are continuous in space, being aware of and responsive to the user’s context and pose. Currently, the time and technical expertise required to create such applications are the main reasons preventing its widespread use. As such, authoring tools which facilitate the development and configuration of pervasive AR experiences have become progressively more relevant. Their operation often involves the navigation of the real-world scene and the use of the AR equipment itself to add the augmented information within the environment. The proposed experimental tool makes use of 3D scans from physical environments to provide a reconstructed digital replica of such spaces for a desktop-based method, and to enable positional tracking for a mobile-based one. While the desktop platform represents a non-immersive setting, the mobile one provides continuous AR in the physical environment. Both versions can be used to place virtual content and ultimately configure an AR experience. The authoring capabilities of the different platforms were compared by conducting a user study focused on evaluating their usability. Although the AR interface was generally considered more intuitive, the desktop platform shows promise in several aspects, such as remote configuration, lower required effort, and overall better scalability.

Hazard recognition and prevention techniques utilize augmented reality (AR) and virtual reality (VR) technology for immersive training and identification of potential hazards. Currently, no scoping reviews exist specifically on hazard recognition and prevention using AR and VR, which is crucial for forming evidence-based guidelines. This scoping review aimed to identify the use of AR and VR in hazard recognition and prevention, identify research gaps, and provide future recommendations. This Joanna Brings Institute (JBI) approach was used to investigate hazard recognition and prevention with AR and VR interventions. A total of six reliable databases were included: Google Scholar, IEEE Xplore, ACM Digital Library, PubMed, Scopus, and APA PsycNet, focusing on records and publications from the last five years. The review identified 5438 publications, with 22 meeting final inclusion criteria. The major findings include the use of AR and VR for teaching new skills, ensuring health and safety, enhancing realism, developing cognitive skills, improving performance, increasing engagement, and stimulating a sense of presence. Limitations identified include limited or non-diverse sample size, absence of real-world scenarios or contextual learning, inadequate game mechanics, insufficient validation of test results, lack of dynamic game behaviour, insufficient expert inclusion, technology issues, long-term impact assessment, and cost comparisons with traditional approaches. Technological issues have emerged as an additional impediment, limiting the efficacy of AR/VR applications in hazard recognition and prevention. In light of these findings, future research should focus on integrating contextual learning, expert inclusion, sample recruitment, expert validation, long-term impact assessment, and resolving technological challenges to enhance AR/VR applications’ efficacy and real-world viability.

In this paper, we propose an approach for supporting the design and implementation of interactive and realistic Augmented Reality (AR). Despite the advances in AR technology, most software applications still fail to support AR experiences where virtual objects appear as merged into the real setting. To alleviate this situation, we propose to combine the use of model-based AR techniques with the advantages of current game engines to develop AR scenes in which the virtual objects collide, are occluded, project shadows and, in general, are integrated into the augmented environment more realistically. To evaluate the feasibility of the proposed approach, we extended an existing game platform named GREP to enhance it with AR capacities. The realism of the AR experiences produced with the software was assessed in an event in which more than 100 people played two AR games simultaneously.

PurposeDrawing on embodied cognition theory, this study examined the impact of midair, gesture-based somatosensory augmented reality (AR) experience on consumer delight and stickiness intention. The mediating effects of three psychological states for body schema (i.e. natural symbol sets, vivid memory and human touch) on the relationships between somatosensory AR and consumer delight/stickiness intention are determined. By filling gaps in the research, we hope to provide guidance on how to drive delightful somatosensory AR marketing.Design/methodology/approachTwo experiments were conducted (Study 1 and Study 2) to test the research model and hypotheses. These experiments compared the effects of the “presence” (midair, gesture-based) and “absence” (mouse-based traditional website) conditions in somatosensory AR on consumer body schema and the creation of a delightful virtual shopping experience (i.e. consumer delight and stickiness intention).FindingsThe consumer delight and stickiness intention created in the presence condition was much higher than those in the absence condition. Consumers appeared to prefer engaging in a midair gesture-based somatosensory AR experience and exploring an augmented metaverse reality to interacting with a mouse-based traditional website. We also found that giving online consumers more somatosensory activities and kinesthetic experiences effectively inspired three psychological states of body schema in online consumers.Originality/valueThe results contribute to the AR experience and somatosensory marketing literature by revealing the role of natural symbol sets, vivid memory and the sense of human touch. This research breaks through the long-developed research paradigm on consumer delight, which has been limited to traditional entities and web contexts. We also extend embodied cognition theory to the study of somatosensory AR marketing.

PurposeDrawing on the stimulus-organism-response (SOR) theory, this study examines the nexus between augmented reality (AR) experiences, emotions and visitors’ attachment. It also explores how visitors’ attachment affects their behaviour towards AR, with consideration of the moderating effect of innovation character.Design/methodology/approachThe study has employed PLS-SEM to analyse 303 responses from tourists who visited the Roman Antonine Baths in Tunisia.FindingsThe results found that AR-related experiences (perceived augmentation and experiential value) positively impact affective responses, which in turn significantly affect attachment to AR. Also, attachment positively influences visitors’ behaviour (proselytism and continuance usage intention). The innovative character did not moderate the link between AR experiences and users’ affective responses.Practical implicationsThe findings offer valuable insights for destination managers, marketers and policymakers on successfully incorporating AR into heritage tourism. Site managers and marketers should prioritize enhancing the quality, interactivity and integration of AR to evoke stronger emotional responses and create more immersive, engaging experiences for visitors at heritage sites.Originality/valueThis study enhances destination marketing literature by developing a model for AR experiences’ impact on heritage tourism, expanding the SOR model to show how AR experiences affect emotions and behaviour at the Antonine Baths in Carthage, Tunisia.