Human-avatar Interaction Research Articles

It’s not all in your feet: Improving penalty kick performance with human-avatar interaction and machine learning

Penalty kicks are increasingly decisive in major international football competitions. Yet, over 30% of shootout kicks are missed. The outcome of the kick often relies on the ability of the penalty taker to exploit anticipatory movements of the goalkeeper to redirect the kick toward the open side of the goal. Unfortunately, this ability is difficult to train using classical methods. We used an augmented reality simulator displaying an holographic goalkeeper to test and train penalty kick performance with 13 young elite players. Machine learning algorithms were used to optimize the learning rate by maintaining an optimal level of training difficulty. Ten training sessions of 20 kicks reduced the redirection threshold by 120ms, which constituted a 28% reduction with respect to the baseline threshold. Importantly, redirection threshold reduction was observed for all trained players, and all things being equal, it corresponded to an estimated 35% improvement of the success rate.

Midfrontal Theta Transcranial Alternating Current Stimulation Facilitates Motor Coordination in Dyadic Human-Avatar Interactions.

Synchronous interpersonal motor interactions require moment-to-moment prediction and proactive monitoring of the partner's actions. Neurophysiologically, this is highlighted by an enhancement of midfrontal theta (4-7 Hz) oscillations. In this study, we explored the causal role of midfrontal theta for interpersonal motor interactions using transcranial alternating current stimulation (tACS). We implemented a realistic human-avatar interaction task in immersive virtual reality where participants controlled a virtual arm and hand to press a button synchronously with a virtual partner. Participants completed the task while receiving EEG-informed theta (Experiment 1) or beta (control frequency, Experiment 2) tACS over the frontal midline, as well as sham stimulation as a control. Results showed that midfrontal theta tACS significantly improved behavioral performance (i.e., reduced interpersonal asynchrony) and participants' motor strategies (i.e., increased movement times and reduced RTs), whereas beta tACS had no effect on these measures. These results suggest that theta tACS over frontal areas facilitates action monitoring and motor abilities supporting interpersonal interactions.

Perspective Taking and Avatar-Self Merging.

Today, avatars often represent users in digital worlds such as in video games or workplace applications. Avatars embody the user and perform their actions in these artificial environments. As a result, users sometimes develop the feeling that their self merges with their avatar. The user realizes that they are the avatar, but the avatar is also the user—meaning that avatar’s appearance, character, and actions also affect their self. In the present paper, we first introduce the event-coding approach of the self and then argue based on the reviewed literature on human-avatar interaction that a self-controlled avatar can lead to avatar-self merging: the user sets their own goals in the virtual environment, plans and executes the avatar’s actions, and compares the predicted with the actual motion outcomes of the avatar. This makes the user feel body ownership and agency over the avatar’s action. Following the event-coding account, avatar-self merging should not be seen as an all-or-nothing process, but rather as a continuous process to which various factors contribute, including successfully taking the perspective of the avatar. Against this background, we discuss affective, cognitive, and visuo-spatial perspective taking of the avatar. As evidence for avatar-self merging, we present findings showing that when users take the avatar’s perspective, they can show spontaneous behavioral tendencies that run counter to their own.

Embodied storytelling and meaning-making at the end of life: VoicingHan avatar life-review for palliative care in cancer patients

ABSTRACT This article presents VoicingHan project as a new form of life-review mediated by digital avatars promoting the reconstruction of self and identity through performativity. Whereas traditional life-review uses interview as primary means of the therapeutic process, VoicingHan is mediated by virtual bodies with self-guided participation in one’s own life stories through embodied storytelling performance. VoicingHan enrolled 12-patients receiving outpatient palliative care at VCU Massey cancer center. The storytelling performances were recorded via avatar video format and distributed to participants for review and/or sharing. The present study considered the avatar videos as qualitative data emerging from these individually constructed life review narratives. In this article, the benefits of VoicngHan life-review were demonstrated by analyzing the avatar video narratives, based on theoretical and developmental frameworks. Through the lens of Social Constructivism, this qualitative study deepens our understanding of human-avatar interactions to engage life review process, within the liminal space of life-threatening illnesses.

Remotely Perceived: Investigating the Influence of Valence on Self-Perception and Social Experience for Dyadic Video-Conferencing With Personalized Avatars

Avatar use on video-conference platforms has found dual purpose in recent times as a potential method for ensuring privacy and improving subjective engagement with remote meeting, provided one can also ensure a minimal loss in the quality of social interaction and sense of personal presence. This work focuses on interactions of this sort through real-time motion captured 3D personalized virtual avatars in a 2D video-conferencing context. Our experiments were designed with the intention of exploring previously defined perceptual illusions that occur with avatar-use in Virtual and Augmented Reality settings, outside of the immersive technological domains where they are normally measured. The research described here was aimed at empirically evaluating three separate dimensions of human-avatar interaction. The first was humans-as-avatars, with experimental conditions that were designed to measure changes to subjective perceptions of self-face ownership and self-concept. The second focus was other-perception, with the unique design of the studies outlined below among the first to measure social presence in a video-call between two human-driven avatars. The third emphasis was on the experiential content involved in avatar use, as there were measurements for emotion induction, fatigue and behavior change included in the data collection. The results describe some evidence for face and body ownership, while participants also reported high levels of social presence with the other avatar, indicating that avatar cameras could be a favorable alternative to non-camera feeds in video conferencing. There were also some useful insights gained regarding emotion elicitation in non-video vs. avatar conditions, as well as avatar-induced behavior change.

Human-Avatar Interaction in Virtual Environment to Assess and Train Sensorimotor: Application to the Slap Shot in Hockey

Here we present the conception, implementation and application of a virtual environment simulator dedicated to the assessment and training of slap shot performance in hockey. The simulator is based on human-avatar interaction, namely a real shooter and a virtual goalkeeper whose behavior is dependent on that of the shooter. The synthesis of the virtual goalkeeper relied on a high-quality model and realistic, motion-captured movements. A regression model based on Kriging was used to predict in real-time the shooter's behavior in order to trigger the blocking moves of the virtual goalkeeper at the right time.
 Our model provided accurate predicted values as well as an estimation of the reliability of these values, which allowed us to optimize the behavioral animation of the virtual goalkeeper. We then ran a validation experiment testing the effectiveness of our simulator. The simulator proved very useful both to assess the initial performance of the players and to train and improve this performance. In particular, training as little as 3 hours with our simulator gave rise to substantial and significant improvements (up to 22 percent) of the redirection threshold, i.e., the minimum time required to successfully redirect a shot during movement execution when the outcome is imperiled. Importantly, in comparison with ‘classical’ training methods, our simulator better triggers (precisely and timely) the movements of the goalkeeper based on the movements of the shooter.

Midline frontal and occipito-temporal activity during error monitoring in dyadic motor interactions

Discrepancies between sensory predictions and action outcome are at the base of error coding. However, these phenomena have mainly been studied focussing on individual performance. Here, we explored EEG responses to motor prediction errors during a human-avatar interaction and show that Theta/Alpha activity of the frontal error-monitoring system works in phase with activity of the occipito-temporal node of the action observation network. Our motor interaction paradigm required healthy individuals to synchronize their reach-to-grasp movements with those of a virtual partner in conditions that did (Interactive) or did not require (Cued) movement prediction and adaptation to the partner's actions. Crucially, in 30% of the trials the virtual partner suddenly and unpredictably changed its movement trajectory thereby violating the human participant's expectation. These changes elicited error-related neuromarkers (ERN/Pe - Theta/Alpha modulations) over fronto-central electrodes during the Interactive condition. Source localization and connectivity analyses showed that the frontal Theta/Alpha activity induced by violations of the expected interactive movements was in phase with occipito-temporal Theta/Alpha activity. These results expand current knowledge about the neural correlates of on-line interpersonal motor interactions linking the frontal error-monitoring system to visual, body motion-related, responses.

Inhibitory Theta Burst Stimulation Highlights the Role of Left aIPS and Right TPJ during Complementary and Imitative Human-Avatar Interactions in Cooperative and Competitive Scenarios.

Competitive and cooperative interactions are based on anticipation or synchronization with the partner's actions. Both forms of interaction may either require performing imitative or complementary movements with respect to those performed by our partner. We explored how parietal regions involved in the control of imitative behavior (temporo-parietal junction, TPJ), goal coding and visuo-motor integration (anterior intraparietal sulcus, aIPS) contribute to the execution of imitative and complementary movements during cooperative and competitive interactions. To this aim, we delivered off-line non-invasive inhibitory brain stimulation to healthy individuals' left aIPS and right TPJ before they were asked to reach and grasp an object together with a virtual partner by either performing imitative or complementary interactions. In different blocks, participants were asked to compete or cooperate with the virtual partner that varied its behavior according to cooperative or competitive contexts. Left aIPS and right TPJ inhibition impaired individuals' performance (i.e., synchrony in cooperative task and anticipation in competition) during complementary and imitative interactions, respectively, in both cooperative and competitive contexts, indicating that aIPS and TPJ inhibition affects own-other action integration and action imitation (that are different in complementary vs imitative interactions) more than action synchronization or anticipation (that are different in cooperative vs competitive contexts).

Tradeoffs in Neuroevolutionary Learning-Based Real-Time Robotic Task Design in the Imprecise Computation Framework

A cyberphysical avatar is a semi-autonomous robot that adjusts to an unstructured environment and performs physical tasks subject to critical timing constraints while under human supervision. This article first realizes a cyberphysical avatar that integrates three key technologies: body-compliant control, neuroevolution, and real-time constraints. Body-compliant control is essential for operator safety, because avatars perform cooperative tasks in close proximity to humans; neuroevolution (NEAT) enables “programming” avatars such that they can be used by non-experts for a large array of tasks, some unforeseen, in an unstructured environment; and real-time constraints are indispensable to provide predictable, bounded-time response in human-avatar interaction. Then, we present a study on the tradeoffs between three design parameters for robotic task systems that must incorporate at least three dimensions: (1) the amount of training effort for robot to perform the task, (2) the time available to complete the task when the command is given, and (3) the quality of the result of the performed task. A tradeoff study in this design space by using the imprecise computation as a framework is to perform a common robotic task, specifically, grasping of unknown objects. The results were validated with a real robot and contribute to the development of a systematic approach for designing robotic task systems that must function in environments like flexible manufacturing systems of the future.

사회 학습 기제로서 IJA와 RJA의 비교: 인간-아바타 머리/손 상호작용을 이용한 연구

Joint Attention (JA) has been known to play a key role in human social learning. However, relative impact of different interaction types has yet to be rigorously examined because of limitation of existing methodologies to simulate human-to-human interaction. In the present study, we designed a new JA paradigm with emulating human-avatar interaction and virtual reality technologies, and tested the paradigm in two experiments with healthy adults. Our results indicated that initiating JA (IJA) condition was more effective than responding JA (RJA) condition for social learning in both head and hand interactions. Moreover, the hand interaction involved better information processing than the head interaction. The implication of the results, the validity of the new paradigm, and limitations of this study were discussed.

Causative role of left aIPS in coding shared goals during human-avatar complementary joint actions.

Successful motor interactions require agents to anticipate what a partner is doing in order to predictively adjust their own movements. Although the neural underpinnings of the ability to predict others' action goals have been well explored during passive action observation, no study has yet clarified any critical neural substrate supporting interpersonal coordination during active, non-imitative (complementary) interactions. Here, we combine non-invasive inhibitory brain stimulation (continuous Theta Burst Stimulation) with a novel human–avatar interaction task to investigate a causal role for higher-order motor cortical regions in supporting the ability to predict and adapt to others' actions. We demonstrate that inhibition of left anterior intraparietal sulcus (aIPS), but not ventral premotor cortex, selectively impaired individuals' performance during complementary interactions. Thus, in addition to coding observed and executed action goals, aIPS is crucial in coding ‘shared goals', that is, integrating predictions about one's and others' complementary actions.

Characterisation of gestural units in light of human-avatar interaction

We present a method for characterizing coverbal gestural units intended for human-avatar interaction. We recorded 12 gesture types, using a motion-capture system. We used the markers positions thus obtained to determine the gestural units after stroke segmentation. We complement our linguistic analysis of gestures with an elaboration of our biomechanical hypotheses, our method of segmentation, our characterization hypotheses and the results obtained.

Quantifying engagement: Measuring player involvement in human–avatar interactions

This research investigated the merits of using an established system for rating behavioral cues of involvement in human dyadic interactions (i.e., face-to-face conversation) to measure involvement in human–avatar interactions. Gameplay audio–video and self-report data from a Feasibility Trial and Free Choice study of an effective peer resistance skill building simulation game (DRAMA–RAMA™) were used to evaluate reliability and validity of the rating system when applied to human–avatar interactions. The Free Choice study used a revised game prototype that was altered to be more engaging. Both studies involved girls enrolled in a public middle school in Central Florida that served a predominately Hispanic (greater than 80%), low-income student population. Audio–video data were coded by two raters, trained in the rating system. Self-report data were generated using measures of perceived realism, predictability and flow administered immediately after game play. Hypotheses for reliability and validity were supported: reliability values mirrored those found in the human dyadic interaction literature. Validity was supported by factor analysis, significantly higher levels of involvement in Free Choice as compared to Feasibility Trial players, and correlations between involvement dimension sub scores and self-report measures. Results have implications for the science of both skill-training intervention research and game design.

An interactive virtual guide for the AR based visit of archaeological sites

One of the most interesting research lines about avatars is the design and the implementation of a synthetic behaviour able to drive avatar's actions according to an adaptive interaction paradigm. This aspect, indeed, is of fundamental importance to many advanced applications involving avatars and humans. This study presents a novel framework exploiting augmented reality to visualize a synthetic 3D virtual guide inside an exhibit or a public gallery, to assist visitors wearing a Head Mounted Display during their visit and providing them with both visual and informative enhancements not available in a standard tour. The Human–avatar interaction is approached through a model based on timed automata to address the conversational issues and to improve the quality of interaction by means of an effective synchronization. A usability study conducted on an application of this research to the “avatar assisted tour” of a roman villa, confirms the efficacy of the approach.

Real-Time Multimodal Human–Avatar Interaction

This paper presents a novel real-time multimodal human-avatar interaction (RTM-HAI) framework with vision-based remote animation control (RAC). The framework is designed for both mobile and desktop avatar-based human-machine or human-human visual communications in real-world scenarios. Using 3-D components stored in the Java mobile 3-D (M3G) file format, the avatar models can be flexibly constructed and customized on the fly on any mobile devices or systems that support the M3G standard. For the RAC head tracker, we propose a 2-D real-time face detection/tracking strategy through an interactive loop, in which the detection and tracking complement each other for efficient and reliable face localization, tolerating extreme user movement. With the face location robustly tracked, the RAC head tracker selects a main user and estimates the user's head rolling, tilting, yawing, scaling, horizontal, and vertical motion in order to generate avatar animation parameters. The animation parameters can be used either locally or remotely and can be transmitted through socket over the network. In addition, it integrates audio-visual analysis and synthesis modules to realize multichannel and runtime animations, visual TTS and real-time viseme detection and rendering. The framework is recognized as an effective design for future realistic industrial products of humanoid kiosk and human-to-human mobile communication.