Mixed Reality Telepresence Research Articles

Improved Real-time 3D Reconstruction Method for Mixed Reality Telepresence

Improved Real-time 3D Reconstruction Method for Mixed Reality Telepresence

Visual Guidance for User Placement in Avatar-Mediated Telepresence Between Dissimilar Spaces.

Rapid advances in technology gradually realize immersive mixed-reality (MR) telepresence between distant spaces. This paper presents a novel visual guidance system for avatar-mediated telepresence, directing users to optimal placements that facilitate the clear transfer of gaze and pointing contexts through remote avatars in dissimilar spaces, where the spatial relationship between the remote avatar and the interaction targets may differ from that of the local user. Representing the spatial relationship between the user/avatar and interaction targets with angle-based interaction features, we assign recommendation scores of sampled local placements as their maximum feature similarity with remote placements. These scores are visualized as color-coded 2D sectors to inform the users of better placements for interaction with selected targets. In addition, virtual objects of the remote space are overlapped with the local space for the user to better understand the recommendations. We examine whether the proposed score measure agrees with the actual user perception of the partner's interaction context and find a score threshold for recommendation through user experiments in virtual reality (VR). A subsequent user study in VR investigates the effectiveness and perceptual overload of different combinations of visualizations. Finally, we conduct a user study in an MR telepresence scenario to evaluate the effectiveness of our method in real-world applications.

Ātea Presence— Enabling Virtual Storytelling, Presence, and Tele-Co-Presence in an Indigenous Setting

<bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Developing, evaluating, and</b> disseminating IT research prototypes for and with indigenous partners is both challenging and rewarding. In conjunction with our domain expert collaborators, Te Rau Aroha Marae [Bluff, Aotearoa/New Zealand (NZ)] and our academic colleagues at the Universities of Waikato and Canterbury, we are implementing a mixed reality telepresence system to connect a diasporic Māori community to their historical, cultural, and geographic mātauranga (knowledge). In this article, we describe our project, tea Presence, which is guided by the principles of partnership, participation, and protection. We describe the design and evaluation of the system developed, the collaborative process we undertook with Te Rau Aroha Marae, and our Māori academic colleagues and report on lessons learned along the way.

A Robust Real-Time 3D Reconstruction Method for Mixed Reality Telepresence

Mixed Reality (MR) is a technology which enable to bring a virtual element into the real-world environment. MR intends to improve reality on the virtual world immerse onto real-world space. Occasionally the MR has been improved as the display technologies advanced progressively. In MR collaborative interface context, the local and remote user work together on collaborative task while sense the immersive environment in the cooperative application. User telepresence is an immersive telepresence, where the reconstruction of a human appears in a real-life. Up till now, producing full telepresence of the life-size human body may require a high transmission bandwidth of the internet. Therefore, this paper explores on a robust real-time 3D reconstruction method for MR telepresence. This paper discusses the previous works on the reconstruction method of a full-body human and the existing research works that have proposed the reconstruction methods for telepresence. Besides the 3D reconstruction method, this paper also enlightens our recent finding on the MR framework to transport a full-body human from a local location to a remote location. The MR telepresence will be discussed, as well as the robust 3D reconstruction method which has been implemented with user telepresence feature where the user experiences an accurate 3D representation of a remote person. The paper ends with the discussion and results, MR telepresence with robust 3D reconstruction method to execute user telepresence.

A Review of Mixed Reality Telepresence

Mixed Reality MR is a technology to make the combination of physical and digital worlds coexist with one another This paper is about to explore the potential MR technology to be used for telepresence Telepresence is a technology to allow user to feel present in specific place so he remotely transferred in a digital representation However in remote collaboration it is crucial to count when a localhost user can seizure and later transmission in a view of their backdrops This paper reviews about the existing works on video conferencing and discovers the many limitations such as limited shared view low resolution and lack of the realism This paper focuses on realism issue and real-time 3D reconstruction methods using depth sensors and provides a review for MR telepresence MR has the ability when users interact with the virtual objects it can admit the understanding of the real world and feel presence in real world when they interact to the surrounding

Effects of the Size of Mixed-Reality Person Representations on Stress and Presence in Telecommunication

We study how mixed reality (MR) telepresence can enhance long-distance human interaction and how altering 3D representations of a remote person can be used to modulate stress and anxiety during social interactions. To do so, we developed an MR telepresence system employing commodity depth sensors and Microsoft’s Hololens. A textured, polygonal 3D model of a person was reconstructed in real time and transmitted over network for rendering in remote location using HoloLens. In this study, we used mock job interview paradigm to induce stress in human–subjects interacting with an interviewer presented as an MR hologram. Participants were exposed to three different types of real-time reconstructed virtual holograms of the interviewer, a natural-sized 3D reconstruction (NR), a miniature 3D reconstruction (SR) and a 2D-display representation (LCD). Participants reported their subjective experience through questionnaires, while their biophysical responses were recorded. We found that the size of 3D representation of a remote interviewer had a significant effect on participants’ stress levels and their sense of presence. The questionnaire data showed that NR condition induced more stress and presence than SR condition and was significantly different from LCD condition. We also found consistent patterns in the biophysical data.

StreamSpace: Pervasive Mixed Reality Telepresence for Remote Collaboration on Mobile Devices

StreamSpace: Pervasive Mixed Reality Telepresence for Remote Collaboration on Mobile Devices

A Mixed Reality Telepresence System for Collaborative Space Operation

This paper presents a mixed reality (MR) system that results from the integration of a telepresence system and an application to improve collaborative space exploration. The system combines free viewpoint video with immersive projection technology to support nonverbal communication (NVC), including eye gaze, interpersonal distance, and facial expression. Importantly, these features can be interpreted together as people move around the simulation, maintaining a natural social distance. The application is a simulation of Mars, within which the collaborators must come to agreement over; for example, where the Rover should land and go. The first contribution is the creation of an MR system supporting contextualization of NVC. Two technological contributions are prototyping a technique to subtract a person from a background that may contain physical objects and/or moving images and a lightweight texturing method for multiview rendering, which provides balance in terms of visual and temporal quality. A practical contribution is the demonstration of pragmatic approaches to sharing space between display systems of distinct levels of immersion. A research tool contribution is a system that allows comparison of conventional authored and video-based reconstructed avatars, within an environment that encourages exploration and social interaction. Aspects of system quality, including the communication of facial expression and end-to-end latency are reported.