Augmented Mirror Research Articles

Offistretch: camera-based real-time feedback for daily stretching exercises

AbstractIn this paper, we present OffiStretch, a camera-based system for optimal stretching guidance at home or in the workplace. It consists of a vision-based method for real-time assessment of the user’s body pose to provide visual feedback as interactive guidance during stretching exercises. Our method compares the users’ actual pose with a pre-trained target pose to assess the quality of stretching for a number of different exercises. We utilize angular and spatial pose features to perform this comparison for each individual exercise. The result of this pose assessment is presented to the user as real-time visual feedback on an "augmented mirror" display. As our method relies simply on a single RGB camera, it can be easily utilized in everyday training scenarios. We validate our method in a user study, comparing users’ performance and motivation in stretching when receiving audio-visual guidance on a TV screen both with and without our live feedback. While participants performed equally well in both conditions, feedback boosted their motivation to perform the exercises, highlighting its potential for increasing users’ well-being. Moreover, our results suggest that participants preferred stretching exercises with our live feedback over the condition without the feedback. Finally, an expert evaluation with professional physiotherapists reveals that further work must target improvements of the feedback to ensure correct guidance during stretching.

Mixed reality guided root canal therapy.

Root canal therapy (RCT) is a widely performed procedure in dentistry, with over 25 million individuals undergoing it annually. This procedure is carried out to address inflammation or infection within the root canal system of affected teeth. However, accurately aligning CT scan information with the patient's tooth has posed challenges, leading to errors in tool positioning and potential negative outcomes. To overcome these challenges, a mixed reality application is developed using an optical see-through head-mounted display (OST-HMD). The application incorporates visual cues, an augmented mirror, and dynamically updated multi-view CT slices to address depth perception issues and achieve accurate tooth localization, comprehensive canal exploration, and prevention of perforation during RCT. Through the preliminary experimental assessment, significant improvements in the accuracy of the procedure are observed. Specifically, with the system the accuracy in position was improved from 1.4 to 0.4 mm (more than a 70% gain) using an Optical Tracker (NDI) and from 2.8 to 2.4 mm using an HMD, thereby achieving submillimeter accuracy with NDI. 6 participants were enrolled in the user study. The result of the study suggests that the average displacement on the crown plane of 1.27 ± 0.83 cm, an average depth error of 0.90 ± 0.72 cm and an average angular deviation of 1.83 ± 0.83°. Our error analysis further highlights the impact of HMD spatial localization and head motion on the registration and calibration process. Through seamless integration of CT image information with the patient's tooth, our mixed reality application assists dentists in achieving precise tool placement. This advancement in technology has the potential to elevate the quality of root canal procedures, ensuring better accuracy and enhancing overall treatmentoutcomes.

Feasibility Study of Using Augmented Mirrors for Alignment Task during Orthopaedic Procedures in Mixed Reality.

Accurate depth estimation poses a significant challenge in egocentric Augmented Reality (AR), particularly for precision-dependent tasks in the medical field, such as needle or tool insertions during percutaneous procedures. Augmented Mirrors (AMs) provide a unique solution to this problem by offering additional non-egocentric viewpoints that enhance spatial understanding of an AR scene. Despite the perceptual advantages of using AMs, their practical utility has yet to be thoroughly tested. In this work, we present results from a pilot study involving five participants tasked with simulating epidural injection procedures in an AR environment, both with and without the aid of an AM. Our findings indicate that using AM contributes to reducing mental effort while improving alignment accuracy. These results highlight the potential of AM as a powerful tool for AR-enabled medical procedures, setting the stage for future exploration involving medical professionals.

When arts and science meet: Digital technology in artistic research

In recent decades, advancements in digital technologies have become a rich source of inspiration for artists, who seek to leave the trodden paths and find novel ways of expression. In addition, digital technologies are increasingly implemented in the development of artistic skills, providing new means to develop the artists’ reflection on their own development. As such, they hold great potential to shape artistic research. Moreover, digital technologies offer possibilities to capture the learning process based on quantitative measurement, thereby becoming a potential interface between artistic and scientific approaches to investigating artistic growth. This contribution presents two artistic projects illustrating the potentialities of the art–science encounter. Embedded in the research paradigm of embodied music cognition, both projects explore the role of the body in music performance (interpretation and improvisation). The first project investigates the relation between gesture and interpretative intentions in a contemporary piano composition. The second project concerns the development of one’s musical language through kinemusical improvisation. A mixed methodology and the use of technology as ‘an augmented mirror’ to monitor artistic practice were applied. Both projects illustrate how the implementation of digital technologies may boost the evolution in artistic research and facilitate novel approaches to music teaching and learning.

Takes Tutu to Ballet

Mirrors have been a core feature in ballet studios for over five hundred years. While physical mirrors provide real-time feedback, they do not inform dancers of their errors. Thus, technologies such as motion tracking have been used to augment what a physical mirror can provide. Current augmented mirrors, however, only implement one mode of communication, usually visual, and do not provide a holistic feedback to dancers that includes all the feedback elements commonly used in ballet classes. We conducted a mixed-method study with 16 novices and 16 expert dancers in which we compared two different modes of communication (visual and verbal), two different types of feedback (value and corrective) and two levels of guidance (mirror, or no mirror). Participants' ballet technique scores were evaluated by a remote teacher on eight ballet combinations (tendue, adagio, pirouette, saute, plié, degage, frappe and battement tendue). We report quantitative and qualitative results that show how the level of guidance, mode of communication, and type of feedback, needs to be tuned in different ways for novices and experts.

Extending MAM5 Meta-Model and JaCalIV E Framework to Integrate Smart Devices from Real Environments.

This paper presents the extension of a meta-model (MAM5) and a framework based on the model (JaCalIVE) for developing intelligent virtual environments. The goal of this extension is to develop augmented mirror worlds that represent a real and virtual world coupled, so that the virtual world not only reflects the real one, but also complements it. A new component called a smart resource artifact, that enables modelling and developing devices to access the real physical world, and a human in the loop agent to place a human in the system have been included in the meta-model and framework. The proposed extension of MAM5 has been tested by simulating a light control system where agents can access both virtual and real sensor/actuators through the smart resources developed. The results show that the use of real environment interactive elements (smart resource artifacts) in agent-based simulations allows to minimize the error between simulated and real system.

Focused augmented mirror based on human visual perception

A mirror is used in various aspects of daily life; thus, most people can use a mirror-metaphor augmented reality (AR) system naturally, without wearing a head-mounted display or AR glasses. An augmented mirror, one of the mirror metaphor AR display, is considered a more appropriate system than video-based virtual mirror display for personalized and immersive interaction in AR because it can show a combined scene in which a rendered virtual world is superimposed on the reflected real world based the viewpoint of a user. In this paper, we propose a focused augmented mirror that implements both viewpoint and depth-of-field matchings. In particular, we design a focused augmented mirror by concentrating on how the depth-of-field influences human visual perception in the augmented mirror system. To compare the differences between the theoretical and practical results, we perform two types of experiments; calculating the focus measure of the combined image photographed at user's viewpoint and evaluating user experience with randomly selected non-expert people. In addition, we suggest an efficient user workspace that is limited by the tracker's geometric configuration and the target object's depth of field. Finally, we describe the physical limitation of our system and propose its solution as a future work.

Manipulating the Experience of Reality for Rehabilitation Applications

Augmented reality (AR) has the potential to change the way therapy and rehabilitation is understood and administered. It can be used to manipulate the experience of reality, resulting in novel rehabilitative applications including but not limited to augmented mirror box (AMB) manipulations. We present a conceptual framework for the effective use of AR in a therapeutic context developed around the aspects of belief, interactivity, predictability, and decoupling. This framework is based on previous work in perception and emotion manipulation and is derived from and illustrated with a number of empirical studies. In particular, we describe how our augmented reflection technology (ART) system is able to manipulate the experience of reality in an effective way and how this demonstrates the potential of augmented environments to improve health and wellbeing.

Beyond the Looking Glass: Fooling the Brain with the Augmented Mirror Box

Video mediated and augmented reality technologies can challenge our sense of what we perceive and believe to be real. Applied appropriately, the technology presents new opportunities for understanding and treating a range of human functional impairments as well as studying the underling psychological bases of these phenomena. This paper describes our augmented mirror box (AMB) technology which builds on the potential of optical mirror boxes by adding functions that can be applied in therapeutic and scientific settings. Here we test hypotheses about limb presence and perception, belief, and pain using laboratory studies to demonstrate proof of concept. The results of these studies provide evidence that the AMB can be used to manipulate beliefs and perceptions and alter the reported experience of pain. We conclude that the system has potential for use in experimental and in clinical settings.