Exocentric Perspective Research Articles

Exploration of exocentric perspective interfaces for virtual reality collaborative tasks

Exocentric views play a pivotal role in computer-mediated collaboration, especially in Collaborative Virtual Environments (CVEs), where focusing on the actions and operations of collaboration partners is crucial. The exocentric perspective offers users a vantage point to ascertain the whereabouts and actions of their partners, enhancing spatial awareness and social presence in CVEs. Moreover, interacting via the Exocentric Perspective Interface (ExPI) can help users complete searching and manipulation tasks remotely and efficiently. This work investigates the potential benefits of two representative ExPIs, World In Miniature (WIM) and 2D Map, for VR collaboration. We conducted a user study with 36 participants (18 pairs) to compare WIM and the 2D Map against a baseline in a VR collaborative task encompassing a series of searching and manipulation tasks with different task complexities (Simple, Medium, and Complex). For the Baseline (BL) condition, participants were not provided with an Exocentric Perspectives Interface (ExPI) but instead were given a map of the virtual environment (VE). The results indicate that these two ExPIs significantly improved task performance, usability, social presence, and user experience while reducing VR sickness. In addition, we also found that WIM outperformed 2D Maps, especially in complex collaborative environments. Based on the findings, three design implications are proposed to guide the design of future VR collaboration systems.

PATRON: Perspective-Aware Multitask Model for Referring Expression Grounding Using Embodied Multimodal Cues

Humans naturally use referring expressions with verbal utterances and nonverbal gestures to refer to objects and events. As these referring expressions can be interpreted differently from the speaker's or the observer's perspective, people effectively decide on the perspective in comprehending the expressions. However, existing models do not explicitly learn perspective grounding, which often causes the models to perform poorly in understanding embodied referring expressions. To make it exacerbate, these models are often trained on datasets collected in non-embodied settings without nonverbal gestures and curated from an exocentric perspective. To address these issues, in this paper, we present a perspective-aware multitask learning model, called PATRON, for relation and object grounding tasks in embodied settings by utilizing verbal utterances and nonverbal cues. In PATRON, we have developed a guided fusion approach, where a perspective grounding task guides the relation and object grounding task. Through this approach, PATRON learns disentangled task-specific and task-guidance representations, where task-guidance representations guide the extraction of salient multimodal features to ground the relation and object accurately. Furthermore, we have curated a synthetic dataset of embodied referring expressions with multimodal cues, called CAESAR-PRO. The experimental results suggest that PATRON outperforms the evaluated state-of-the-art visual-language models. Additionally, the results indicate that learning to ground perspective helps machine learning models to improve the performance of the relation and object grounding task. Furthermore, the insights from the extensive experimental results and the proposed dataset will enable researchers to evaluate visual-language models' effectiveness in understanding referring expressions in other embodied settings.

Designing Gaze-Based Interactions for Teleoperation: Eye Stick and Eye Click

Perspective-taking and attentional switching are some of the ergonomic challenges that existing teleoperation human-machine interface designs need to address. This study developed two gaze interaction methods, the Eye Stick and the Eye Click, which were based on the joystick metaphor and the navigation metaphor, respectively, to be used in exocentric perspective teleoperation scenarios. We conducted two user studies to test the task performance and the subjective experience of the gaze interaction methods in a virtual ground vehicle teleoperation task. The results showed that compared with a traditional joystick design, the Eye Stick led to a shorter driving distance and the Eye Click led to less task time, and the gaze interaction methods had performance advantages in more difficult mazes. After multiple task sessions, the participants reported that the gaze interaction methods and the traditional joystick were similar in terms of task workload, perceived learnability, and satisfaction; however, the perceived usability of the Eye Stick was not as good as the Eye Click and the traditional joystick. In conclusion, both the Eye Stick and the Eye Click are feasible and promising gaze interaction methods for teleoperation applications with task performance advantages; however, more research is needed to optimize their user experience design.

Holographic Feature Learning of Egocentric-Exocentric Videos for Multi-Domain Action Recognition

Though existing cross-domain action recognition methods successfully improve the performance on videos of one view ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.</i> , egocentric videos) by transferring the knowledge from videos of another view ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.</i> , exocentric videos), they have limitations in generality because the source and target domains need to be fixed aforehand. In this paper, we propose to solve a more practical task of multi-domain action recognition on egocentric-exocentric videos, which aims to learn a single model to recognize test videos from either egocentric perspective or exocentric perspective by transferring knowledge between two domains. Though previous cross-domain methods can also transfer knowledge from one domain to another one by learning view-invariant representations of two video domains, they are not suitable for the multi-domain action recognition task because they always suffer from the problem of losing view-specific visual information. As a solution to the multi-domain action recognition task, we propose to map a video from either egocentric perspective or exocentric perspective to a global feature space (we call it holographic feature space) that shares both view-invariant and view-specific visual knowledge of two views. Specially, we decompose the video feature into view-invariant component and view-specific component, where view-specific component is written into memory networks for saving view-specific visual knowledge. The final holographic feature combines view-invariant feature and view-specific features of two views based on the memory networks. We demonstrate the effectiveness of the proposed method with extensive experimental results on two public datasets. Moreover, the good performances under the semi-supervised setting show the generality of our model.

Two Decades of Touchable and Walkable Virtual Reality for Blind and Visually Impaired People: A High-Level Taxonomy

Although most readers associate the term virtual reality (VR) with visually appealing entertainment content, this technology also promises to be helpful to disadvantaged people like blind or visually impaired people. While overcoming physical objects’ and spaces’ limitations, virtual objects and environments that can be spatially explored have a particular benefit. To give readers a complete, clear and concise overview of current and past publications on touchable and walkable audio supplemented VR applications for blind and visually impaired users, this survey paper presents a high-level taxonomy to cluster the work done up to now from the perspective of technology, interaction and application. In this respect, we introduced a classification into small-, medium- and large-scale virtual environments to cluster and characterize related work. Our comprehensive table shows that especially grounded force feedback devices for haptic feedback (‘small scale’) were strongly researched in different applications scenarios and mainly from an exocentric perspective, but there are also increasingly physically (‘medium scale’) or avatar-walkable (‘large scale’) egocentric audio-haptic virtual environments. In this respect, novel and widespread interfaces such as smartphones or nowadays consumer grade VR components represent a promising potential for further improvements. Our survey paper provides a database on related work to foster the creation process of new ideas and approaches for both technical and methodological aspects.

F-Formations for Social Interaction in Simulation Using Virtual Agents and Mobile Robotic Telepresence Systems

F-formations are a set of possible patterns in which groups of people tend to spatially organize themselves while engaging in social interactions. In this paper, we study the behavior of teleoperators of mobile robotic telepresence systems to determine whether they adhere to spatial formations when navigating to groups. This work uses a simulated environment in which teleoperators are requested to navigate to different groups of virtual agents. The simulated environment represents a conference lobby scenario where multiple groups of Virtual Agents with varying group sizes are placed in different spatial formations. The task requires teleoperators to navigate a robot to join each group using an egocentric-perspective camera. In a second phase, teleoperators are allowed to evaluate their own performance by reviewing how they navigated the robot from an exocentric perspective. The two important outcomes from this study are, firstly, teleoperators inherently respect F-formations even when operating a mobile robotic telepresence system. Secondly, teleoperators prefer additional support in order to correctly navigate the robot into a preferred position that adheres to F-formations.

Cockpit Displays of Traffic and Weather Information: Effects of 3D Perspective Versus 2D Coplanar Rendering and Database Integration

ABSTRACTObjective: The objectives were to evaluate the viability of 3-dimensional (3D) perspective aircraft hazard displays for traffic and weather, in isolation, and when the two were integrated into a single display panel.Background: Prior research identified the costs of such 3D perspective hazard displays, presented from an exocentric perspective, because of their ambiguity in presenting spatial relations in an unstructured 3D space. Research also supported the advantage of display overlay, particularly when both databases (traffic and weather) need to be integrated to make hazard avoidance judgments. The two variables had not been examined in conjunction.Method: Seventeen certified flight instructors flew a series of hazard avoidance maneuvers in a part-task simulator, with four displays, created by the combination of 3D versus 2-dimensional (2D) perspective and separate versus overlay hazard formats. Positioning of traffic and weather that would make it necessary to avoid hazards was designed to allow the study of both the strengths of overlay and weaknesses of 3D integration.Results: The data clearly indicated that hazard avoidance was worse with the 3D perspective display, for both traffic and weather hazards, and pointed to ambiguity as the cause. Display overlay was better than separation, but only when the task required consulting both databases.Conclusion: 3D perspective displays are not advised for hazard presentation on the flight deck. Overlays can be helpful, as long as clutter issues are addressed.

Transcranial direct current stimulation of the right temporoparietal junction impairs third-person perspective taking

Given the current debates about the precise functional role of the right temporoparietal junction (rTPJ) in egocentric and exocentric perspective taking, in the present study we manipulated activity in the rTPJ to investigate the effects on a spatial perspective-taking task. Participants engaged in a mental body transformation task, requiring them to mentally rotate their own body to the position of an avatar, while undergoing anodal, cathodal, or sham transcranial direct current stimulation (tDCS) of the rTPJ. As a control task, participants judged the laterality of a stimulus feature with respect to a fixation cross on the screen. For the first half of the experiment (only during online tDCS), a task-selective effect of tDCS was observed, reflected in slower reaction times following anodal than following cathodal and sham tDCS for the mental body transformation task, but not for the control task. The effects of tDCS were most pronounced for stimuli implying a more difficult mental body transformation. No effects of tDCS were observed during the second half of the experiment. The effects of tDCS were most pronounced for participants scoring low on aberrant perceptual beliefs and spiritual transcendence, suggesting a relation between third-person perspective taking and bodily and perceptual experiences. The finding that anodal stimulation of the rTPJ impairs third-person perspective taking indicates a key role of this region in exocentric spatial processing.

Facilitating Perspective Taking in Groups

The present study investigates the nature of egocentric biases in a situation where a speaker is surrounded by social actors with different perspectives. In this context, the author investigated how communication channels function to ease egocentric bias during collaborative activities. To investigate this point, the author used conversational agents as social actors. The present study therefore created a virtual situation where a speaker was surrounded by several speakers. The author hypothesized that the diversity of communication channels available to the audience would increase the awareness of others and facilitate the adoption of an exocentric perspective. The results of the analysis show that participants who engaged in the collaboration task with various communication channels used fewer egocentric perspectives. Studies in egocentrism and communication have not yet investigated the conversational dynamics of multiple speakers. This study therefore provides a new perspective about the kinds of factors that may ease such biases.

Spatial learning in a virtual multilevel building: Evaluating three exocentric view aids

The present study explores how the design of the exocentric view aid affects the acquisition of survey knowledge in virtual environments. The exocentric view was provided by either a 3D floor map, a 3D building map or the elevation of viewpoint in air. Participants navigated a virtual multilevel building and their survey knowledge was measured by the judgment of spatial relative direction. The results showed that (1) the accuracy of spatial judgment along the horizontal direction and response time were improved for participants with the exocentric view aid; (2) the accuracy of spatial judgment along the vertical direction was worst in the condition with a 3D floor map; (3) in general participants with a 3D building map performed best. The data suggested that the large scale of an exocentric view aid and the increased number of exocentric perspective through which the spatial layout is observed can facilitate the acquisition of survey knowledge in a virtual building. Potential applications of the findings include the design of a 3D map for navigation in both real and virtual buildings.

Preclinical evaluation of an augmented reality system for craniofacial surgery

The MagicBook is a Mixed Reality interface that uses a real book to seamlessly transport users between Reality and Virtuality. A vision-based tracking method is used to overlay virtual models on real book pages, creating an Augmented Reality (AR) scene. When users see an AR scene they are interested in they can fly inside it and experience it as an immersive Virtual Reality (VR). The interface also supports multi-scale collaboration, allowing multiple users to experience the same virtual environment either from an egocentric or an exocentric perspective. In this paper we describe the MagicBook prototype, potential applications and user feedback.

The MagicBook: a transitional AR interface

The MagicBook is a Mixed Reality interface that uses a real book to seamlessly transport users between Reality and Virtuality. A vision-based tracking method is used to overlay virtual models on real book pages, creating an Augmented Reality (AR) scene. When users see an AR scene they are interested in they can fly inside it and experience it as an immersive Virtual Reality (VR). The interface also supports multi-scale collaboration, allowing multiple users to experience the same virtual environment either from an egocentric or an exocentric perspective. In this paper we describe the MagicBook prototype, potential applications and user feedback.

Frames of reference for the display of battlefield information: judgment-display dependencies.

In 2 experiments, U.S. Army soldiers viewed computer-generated displays that presented battlefield information from 3 different frames of reference: a 2D plan view display (with contour lines), a 3D exocentric perspective display, and an interactive 3D immersed display. In Experiment 1, soldiers made geographical judgments. The results suggested that both 3D displays suffered from ambiguity of distance estimates but that the 3D immersed display was most accurate for judging whether a location is directly visible from another. In Experiment 2, the 3D exocentric display was compared with a 3D immersed view, coupled with a small 2D inset map, in a more continuous battlefield scenario in which judgments of enemy activity were made. The findings of 3D ambiguity were replicated from Experiment 1. The accuracy of judgments of enemy activity suffered with the immersed display when information necessary to answer correctly did not appear in the initial forward view and required panning to acquire, reflecting the cognitive demands of integration across different views. This display also hindered soldiers' ability to report changes in enemy activity from one scene to the next. The results of this research will help to provide guidelines for the appropriate choice of computer display technology to assist in designing battlefield visualization aids. Caution should be exercised in choosing immersive viewpoints.

Frame of Reference and Navigation Through Document Visualizations: Flying Through Information Space

An experiment was conducted on navigation through three-dimensional visualizations of document databases. Subjects navigated through a 1000-document information space in which documents near each other were likely to be about a similar topic and keywords displayed in the space indicated the topic of nearby documents. The influence of frame of reference and use of a map on document search and spatial judgment tasks was investigated. The frame of reference distinction compared a more egocentric (inside-out, immersed) perspective and a more exocentric (outside-in, bird's eye) perspective. Performance was best with the exocentric display, with the egocentric display a close second. Use of a map hindered rather than supported performance. Results revealed tool/strategy choice as an important mediator of the influence of frame of reference, with non-optimal tool choices observed. Spatial ability correlated with performance. Results are accounted for in terms of mediating influences.