Embodiment In Virtual Reality Research Articles

Effects of a virtual gender swap on social and temporal decision-making

Mounting evidence has demonstrated that embodied virtual reality, during which physical bodies are replaced with virtual surrogates, can strongly alter cognition and behavior even when the virtual body radically differs from one’s own. One particular emergent area of interest is the investigation of how virtual gender swaps can influence choice behaviors. Economic decision-making paradigms have repeatedly shown that women tend to display more prosocial sharing choices than men. To examine whether a virtual gender swap can alter gender-specific differences in prosociality, 48 men and 51 women embodied either a same- or different-gender avatar in immersive virtual reality. In a between-subjects design, we differentiated between specifically social and non-social decision-making by means of a virtually administered interpersonal and intertemporal discounting task, respectively. We hypothesized that a virtual gender swap would elicit social behaviors that stereotypically align with the gender of the avatar. To relate potential effects to changes in self-perception, we also measured implicit and explicit identification with gendered (or gender-typical) traits prior to and following the virtual experience, and used questionnaires that assessed the strength of the illusion. Contrary to our hypothesis, our results show that participants made less prosocial decisions (i.e., became more selfish) in different-gender avatars, independent of their own biological sex. Moreover, women embodying a male avatar in particular were more sensitive to temptations of immediate rewards. Lastly, the manipulation had no effects on implicit and explicit identification with gendered traits. To conclude, while we showed that a virtual gender swap indeed alters decision-making, gender-based expectancies cannot account for all the task-specific interpersonal and intertemporal changes following the virtual gender swap.

Identifying research gaps: A review of virtual patient education and self-management.

Avatars in Virtual Reality (VR) can not only represent humans, but also embody intelligent software agents that communicate with humans, thus enabling a new paradigm of human-machine interaction. The research agenda proposed in this paper by an interdisciplinary team is motivated by the premise that a conversation with a smart agent avatar in VR means more than giving a face and body to a chatbot. Using the concrete communication task of patient education, this research agenda is rather intended to explore which patterns and practices must be constructed visually, verbally, para- and nonverbally between humans and embodied machines in a counselling context so that humans can integrate counselling by an embodied VR smart agent into their thinking and acting in one way or another. The scientific literature in different bibliographical databases was reviewed. A qualitative narrative approach was applied for analysis. A research agenda is proposed which investigates how recurring consultations of patients with healthcare professionals are currently conducted and how they could be conducted with an embodied smart agent in immersive VR. Interdisciplinary teams consisting of linguists, computer scientists, visual designers and health care professionals are required which need to go beyond a technology-centric solution design approach. Linguists' insights from discourse analysis drive the explorative experiments to identify test and discover what capabilities and attributes the smart agent in VR must have, in order to communicate effectively with a human being.

Using a Fully Expressive Avatar to Collaborate in Virtual Reality: Evaluation of Task Performance, Presence, and Attraction

Avatar-mediated collaboration in virtual environments is becoming more and more prevalent. However, current consumer systems are not suited to fully replicate real-world nonverbal communication. We present a novel avatar system for collaboration in virtual reality, which supports high levels of nonverbal expression by tracking behavior such as body movement, hand gesture, and facial expression. The system was built using camera tracking technology only. Therefore, in contrast to many other high-level tracking systems, it does not require users to wear additional trackers on their bodies. We compared our highly expressive system with a consumer setup extended with two body-worn trackers in a dyadic study. We investigated users’ performance, such as completion time and accuracy, as well as the presence and interpersonal attraction in a virtual charades game using an asymmetric control scheme. The results show that participants interacting with highly expressive avatars felt more social presence and attraction and exhibited better task performance than those interacting with partners represented using low-expressive avatars. Hence, we conclude that virtual reality avatar systems benefit from a higher level of nonverbal expressiveness, which can be achieved without additional body-worn trackers.

A multi-modal modified feedback self-paced BCI to control the gait of an avatar

Brain–computer interfaces (BCIs) have been used to control the gait of a virtual self-avatar with a proposed application in the field of gait rehabilitation. Some limitations of existing systems are: (a) some systems use mental imagery (MI) of movements other than gait; (b) most systems allow the user to take single steps or to walk but do not allow both; (c) most function in a single BCI mode (cue-paced or self-paced). Objective. The objective of this study was to develop a high performance multi-modal BCI to control single steps and forward walking of an immersive virtual reality avatar. Approach. This system used MI of these actions, in cue-paced and self-paced modes. Twenty healthy participants participated in this study, which was comprised of four sessions across four different days. They were cued to imagine a single step forward with their right or left foot, or to imagine walking forward. They were instructed to reach a target by using the MI of multiple steps (self-paced switch-control mode) or by maintaining MI of forward walking (continuous-control mode). The movement of the avatar was controlled by two calibrated regularized linear discriminate analysis classifiers that used the µ power spectral density over the foot area of the motor cortex as a feature. The classifiers were retrained after every session. For a subset of the trials, positive modified feedback (MDF) was presented to half of the participants, where the avatar moved correctly regardless of the classification of the participants’ MI. The performance of the BCI was computed on each day, using different control modes. Main results. All participants were able to operate the BCI. Their average offline performance, after retraining the classifiers was 86.0 ± 6.1%, showing that the recalibration of the classifiers enhanced the offline performance of the BCI (p < 0.01). The average online performance was 85.9 ± 8.4% showing that MDF enhanced BCI performance (p = 0.001). The average performance was 83% at self-paced switch control and 92% at continuous control mode. Significance. This study reports on a first BCI to use motor imagery of the lower limbs in order to control the gait of an avatar with different control modes and different control commands (single steps or forward walking). BCI performance is increased in a novel way by combining three different performance enhancement techniques, resulting in a single high performance and multi-modal BCI system. This study also showed that the improvements due to the effects of MDF lasted for more than one session.

Study of Full-body Virtual Embodiment Using noninvasive Brain Stimulation and Imaging

ABSTRACT The sense of embodiment in virtual reality is a strong case of body ownership illusion, effectively allowing humans to experience the ownership of a modified, or a completely different body. Virtual embodiment has captured the attention of researchers in various fields, with applications far beyond computer science. Despite the promising applications, little is known about the neural mechanisms behind full-body virtual embodiment. This study investigates the influence of anodal transcranial direct current stimulation of the brain area linked to processing of the bodily self (right temporoparietal junction) to the subjective strength of virtual embodiment and its main constituents, using within-subject experimental design with sham-controlled stimulation. Virtual embodiment was studied using questionnaires, accompanied by brain signals gathered using EEG. Our results suggest that stimulation did not affect the sense of ownership toward the virtual avatar. Borderline strengthening of the perceived sense of agency toward the avatar’s actions was found in the sessions with stimulation.

Embodiment in Virtual Reality

Virtual reality (VR) has been a prominent idea for exploring new worlds beyond the physical, and in recent decades, it has evolved in many aspects. The notion of immersion and the sense of presence in VR gained new definitions as technological advances took place. However, even today, we can question whether the degrees of immersion achieved through this technology are profound and felt. A fundamental aspect is the sense of embodiment in the virtual space. To what extent do we feel embodied in virtual environments? In this publication, the authors present works that challenge and question the embodiment sensation in VR, specifically in the artistic aspect. They present initial reflections about embodiment in virtuality and analyze the technologies adopted in creating interactive artworks prepared for galleries and theater stage, questioning the sensations caused by the visual embodiment in virtual reality under the perspective of both the audience and the performer.

Role of Kinematics Assessment and Multimodal Sensorimotor Training for Motion Deficits in Breast Cancer Chemotherapy-Induced Polyneuropathy: A Perspective on Virtual Reality Avatars.

Chemotherapy-induced polyneuropathy (CIPN), one of the most severe and incapacitating side effects of chemotherapeutic drugs, is a serious concern in breast cancer therapy leading to dose diminution, delay, or cessation. The reversibility of CIPN is of increasing importance since active chemotherapies prolong survival. Clinical assessment tools show that patients experiencing sensorimotor CIPN symptoms not only do they have to cope with loss in autonomy and life quality, but CIPN has become a key restricting factor in treatment. CIPN incidence poses a clinical challenge and has lacked established and efficient therapeutic options up to now. Complementary, non-opioid therapies are sought for both prevention and management of CIPN. In this perspective, we explore the potential that digital interventions have for sensorimotor CIPN rehabilitation in breast cancer patients. Our primary goal is to emphasize the benefits and impact that Virtual Reality (VR) avatars and Machine Learning have in combination in a digital intervention aiming at (1) assessing the complete kinematics of deficits through learning underlying patient sensorimotor parameters, and (2) parameterize a multimodal VR simulation to drive personalized deficit compensation. We support our perspective by evaluating sensorimotor effects of chemotherapy, the metrics to assess sensorimotor deficits, and relevant clinical studies. We subsequently analyse the neurological substrate of VR sensorimotor rehabilitation, with multisensory integration acting as a key element. Finally, we propose a closed-loop patient-centered design recommendation for CIPN sensorimotor rehabilitation. Our aim is to provoke the scientific community toward the development and use of such digital interventions for more efficient and targeted rehabilitation.

The eyes have it

Interacting with people across large distances is important for remote work, interpersonal relationships, and entertainment. While such face-to-face interactions can be achieved using 2D video conferencing or, more recently, virtual reality (VR), telepresence systems currently distort the communication of eye contact and social gaze signals. Although methods have been proposed to redirect gaze in 2D teleconferencing situations to enable eye contact, 2D video conferencing lacks the 3D immersion of real life. To address these problems, we develop a system for face-to-face interaction in VR that focuses on reproducing photorealistic gaze and eye contact. To do this, we create a 3D virtual avatar model that can be animated by cameras mounted on a VR headset to accurately track and reproduce human gaze in VR. Our primary contributions in this work are a jointly-learnable 3D face and eyeball model that better represents gaze direction and upper facial expressions, a method for disentangling the gaze of the left and right eyes from each other and the rest of the face allowing the model to represent entirely unseen combinations of gaze and expression, and a gaze-aware model for precise animation from headset-mounted cameras. Our quantitative experiments show that our method results in higher reconstruction quality, and qualitative results show our method gives a greatly improved sense of presence for VR avatars.

O29 Development of a Virtual Reality Avatar Software Program to Prevent Obesity in Children From Low-Income Communities

Background Many families from low-income communities lack access to evidence-based nutrition education opportunities. Thus, we developed a virtual reality avatar program to provide parents and children with tailored nutrition education on healthy snacking. Objective To assess the effectiveness, usability, and applicability of an innovative virtual reality avatar software to improve healthy snacking behaviors in children. Study Design, Setting, Participants Qualitative methods were used for product development user testing. The research focused on children (ages 5-10) from low-income communities in North Carolina. First, children from a youth food summit provided anonymous written feedback on their reactions to the program. Second, this initial feedback was used to modify the program, which was then given to parent-child dyads (n = 15) who were observed using the program and then completed an in-depth semi-structured interview about their experiences. Eligibility for the second phase included being English speaking, low-income (federal benefit eligible), and having a child between ages 5-10. All participants for the observations/interviews were African American and received a gift card for their participation. Measurable Outcome/Analysis Interest in using the program, usability, and overall perceptions were determined with a Thematic analysis approach in Atlas Ti. Results The anonymous written feedback from children was mostly positive. Most children found the program to be “cool,” “fun,” and “helpful.” Children suggested changing the voice of the avatar, adding more pictures and interaction (eg, gamifying), and allowing personalization of the avatar. In the interviews, children stated they were excited about interacting with the avatar and parents liked that it modeled healthy behaviors. The program helped to foster dialogue between children and parents, and the avatar's probes helped parents learn new information about their children's preferences/behaviors. Conclusion The virtual reality avatar program appears to be a promising nutrition education approach for children. Evaluation of impact on dietary behaviors is needed. Funding Health Resources and Services Administration.

Gait coordination in overground walking with a virtual reality avatar.

Little information is currently available on interpersonal gait synchronization in overground walking. This is caused by difficulties in continuous gait monitoring over many steps while ensuring repeatability of experimental conditions. These challenges could be overcome by using immersive virtual reality (VR), assuming it offers ecological validity. To this end, this study provides some of the first evidence of gait coordination patterns for overground walking dyads in VR. Six subjects covered the total distance of 27 km while walking with a pacer. The pacer was either a real human subject or their anatomically and biomechanically representative VR avatar driven by an artificial intelligence algorithm. Side-by-side and front-to-back arrangements were tested without and with the instruction to synchronize steps. Little evidence of spontaneous gait coordination was found in both visual conditions, but persistent gait coordination patterns were found in the case of intentional synchronization. Front-to-back rather than side-by-side arrangement consistently yielded in the latter case higher mean synchronization strength index. Although the mean magnitude of synchronization strength index was overall comparable in both visual conditions when walking under the instruction to synchronize steps, quantitative and qualitative differences were found which might be associated with common limitations of VR solutions.

Eye fixations and electrodermal activity during low‐budget virtual reality embodiment

SummaryAs motion‐sensing devices become more accessible to consumers, it is important to understand how users experience embodiment when using these devices. In our 3 × 2 between‐groups study, we explored eye fixations and electrodermal activity (EDA) in order to more objectively understand potential interaction effects between the self‐avatar body, and the presence of a mirror within the context of low‐budget embodiment. We developed six experimental conditions concerning Body (human, mannequin, and zombie self‐avatars) and Mirror (mirror and no mirror) factors, and presented participants with a virtual environment in which they could control their self‐avatars by using HTC Vive controllers and trackers. In addition to eye fixations and EDA, we assessed self‐reported data concerning body ownership, agency, self‐location, as well as enjoyment of the experience. Our results suggest that the Body may have been more influential in eliciting body ownership than the Mirror, and that an interaction effect between Gender and Body may influence eye gaze behavior. Additionally, female participants reported significantly higher agency than males. We consider logical next steps for similar research which might elaborate upon our findings.

Eye contact in virtual reality – A psychophysiological study

This experiment investigated whether eye contact would evoke similar attention and emotion related psychophysiological responses in virtual reality (VR) as in a face-to-face interaction. Participants viewed a confederate in a live interaction (Live condition) and a confederate's avatar in VR (VR condition). In both conditions, the confederate/avatar was portraying direct and laterally averted gaze. Heart rate deceleration responses reflecting attention orienting were greater to direct gaze compared to averted gaze, and the effect was not significantly different between Live and VR conditions. However, skin conductance responses reflecting physiological arousal were larger in response to direct than averted gaze only in the Live condition. These results suggest that while eye contact with a live person evokes substantial attention and emotion related psychophysiological responses, the physiological effects of eye contact are diminished in VR.

T227. VIRTUAL REALITY SOCIAL SKILLS TRAINING IMPROVES SOCIAL ATTENTION IN INDIVIDUALS WITH SCHIZOPHRENIA

BackgroundSocial impairment is a core feature of schizophrenia presenting a major barrier to recovery. Although antipsychotic medications can reduce psychotic symptoms, social impairments often persist, contributing to poor outcome. Validated interventions, such as Social Skills Training (Bellack et al., 2004), target a broad range of social domains but they yield only modest effect sizes for social outcome (Pfammatter et al, 2006). Moreover, conventional social interventions suffer from high burden on the clients and therapists, low adherence, lack of personalization, and low generalizability. Importantly, social interventions are not widely available. Virtual reality (VR) offers a viable alternative to conventional therapies with several advantages including high acceptability, an extensive repertoire of stimuli, low-burden, low-cost and safety (Strickland, 1997). Importantly, VR allows for a precise targeting of social cognitive mechanisms. Social attention, the fast orientation and allocation of resources to social stimuli can be indexed by tracking eye movements. Abnormal eye scanning behavior during social interactions has been linked to poor social functioning in schizophrenia (Brunet-Gouet & Decety, 2006). We designed a VR-based social skills training game to directly target social attention in schizophrenia.MethodsEighteen individuals with schizophrenia (SZ) participated in the VR training game twice a week for 5 weeks. Eye movement patterns were recorded throughout the training. Subjects were required to solve social “missions” (e.g., obtain personal information through conversations with avatars) in different naturalistic scenarios. To start a mission, participants had to fixate on the chosen avatar. The latency to engage in a social interaction was indexed by the fixation time (social engagement latency). Each session consisted of 12 missions. We compared social engagement latency and eye gaze patterns pre- and post-training to assess the efficacy of the VR social skills training program.ResultsSocial engagement latency significantly decreased after 10 sessions (d=0.78). This result suggests that prosocial attention benefitted from VR training. We also found that the standard deviation of dwell time (i.e., proportion of time spent looking at the avatar’s face during a conversation) significantly increased across training sessions (d=0.56). This result suggests an increased modulation of interpersonal engagement during social interaction. Additionally, participants’ emotion recognition ability significantly increased (η2 = 0.27), and negative symptoms significantly decreased (η2=0.34) from pre- to post- training. Importantly, these changes in social attention correlated with improvement in negative symptoms.DiscussionSimulated and targeted social interactions with avatars in VR significantly improved social attention in individuals with schizophrenia. Importantly, improvements in negative symptoms and emotion perception after training suggest that this training protocol has an impact on a broad range of social functions. VR training is a promising alternative to traditional psychosocial interventions to target specific mechanisms underlying social functioning in schizophrenia.

Mechanisms of the breathing contribution to bodily self-consciousness in healthy humans: Lessons from machine-assisted breathing?

Previous studies investigated bodily self‐consciousness (BSC) by experimentally exposing subjects to multisensory conflicts (i.e., visuo‐tactile, audio‐tactile, visuo‐cardiac) in virtual reality (VR) that involve the participant's torso in a paradigm known as the full‐body illusion (FBI). Using a modified FBI paradigm, we found that synchrony of visuo‐respiratory stimulation (i.e., a flashing outline surrounding an avatar in VR; the flash intensity depending on breathing), is also able to modulate BSC by increasing self‐location and breathing agency toward the virtual body. Our aim was to investigate such visuo‐respiratory effects and determine whether respiratory motor commands contributes to BSC, using non‐invasive mechanical ventilation (i.e., machine‐delivered breathing). Seventeen healthy participants took part in a visuo‐respiratory FBI paradigm and performed the FBI during two breathing conditions: (a) “active breathing” (i.e., participants actively initiate machine‐delivered breaths) and (b) “passive breathing” (i.e., breaths’ timing was determined by the machine). Respiration rate, tidal volume, and their variability were recorded. In line with previous results, participants experienced subjective changes in self‐location, breathing agency, and self‐identification toward the avatar's body, when presented with synchronous visuo‐respiratory stimulation. Moreover, drift in self‐location was reduced and tidal volume variability were increased by asynchronous visuo‐respiratory stimulations. Such effects were not modulated by breathing control manipulations. Our results extend previous FBI findings showing that visuo‐respiratory stimulation affects BSC, independently from breathing motor command initiation. Also, variability of respiratory parameters was influenced by visuo‐respiratory feedback and might reduce breathing discomfort. Further exploration of such findings might inform the development of respiratory therapeutic tools using VR in patients.

Timing and correction of stepping movements with a virtual reality avatar.

Research into the ability to coordinate one’s movements with external cues has focussed on the use of simple rhythmic, auditory and visual stimuli, or interpersonal coordination with another person. Coordinating movements with a virtual avatar has not been explored, in the context of responses to temporal cues. To determine whether cueing of movements using a virtual avatar is effective, people’s ability to accurately coordinate with the stimuli needs to be investigated. Here we focus on temporal cues, as we know from timing studies that visual cues can be difficult to follow in the timing context. Real stepping movements were mapped onto an avatar using motion capture data. Healthy participants were then motion captured whilst stepping in time with the avatar’s movements, as viewed through a virtual reality headset. The timing of one of the avatar step cycles was accelerated or decelerated by 15% to create a temporal perturbation, for which participants would need to correct to, in order to remain in time. Step onset times of participants relative to the corresponding step-onsets of the avatar were used to measure the timing errors (asynchronies) between them. Participants completed either a visual-only condition, or auditory-visual with footstep sounds included, at two stepping tempo conditions (Fast: 400ms interval, Slow: 800ms interval). Participants’ asynchronies exhibited slow drift in the Visual-Only condition, but became stable in the Auditory-Visual condition. Moreover, we observed a clear corrective response to the phase perturbation in both the fast and slow tempo auditory-visual conditions. We conclude that an avatar’s movements can be used to influence a person’s own motion, but should include relevant auditory cues congruent with the movement to ensure a suitable level of entrainment is achieved. This approach has applications in physiotherapy, where virtual avatars present an opportunity to provide the guidance to assist patients in adhering to prescribed exercises.

Using Facial Animation to Increase the Enfacement Illusion and Avatar Self-Identification.

Through avatar embodiment in Virtual Reality (VR) we can achieve the illusion that an avatar is substituting our body: the avatar moves as we move and we see it from a first person perspective. However, self-identification, the process of identifying a representation as being oneself, poses new challenges because a key determinant is that we see and have agency in our own face. Providing control over the face is hard with current HMD technologies because face tracking is either cumbersome or error prone. However, limited animation is easily achieved based on speaking. We investigate the level of avatar enfacement, that is believing that a picture of a face is one's own face, with three levels of facial animation: (i) one in which the facial expressions of the avatars are static, (ii) one in which we implement lip-sync motion and (iii) one in which the avatar presents lip-sync plus additional facial animations, with blinks, designed by a professional animator. We measure self-identification using a face morphing tool that morphs from the face of the participant to the face of a gender matched avatar. We find that self-identification on avatars can be increased through pre-baked animations even when these are not photorealistic nor look like the participant.

Exploration of the Value of Facially Expressive Avatars with VR Developers

Ayfer Gokalp and Jacqueline Pospisil are Facebook user experience researchers from Seattle, Washington, United States, and they focus on the development of software and hardware products within the virtual reality (VR) space. For this research, they explored the value of a facially expressive avatars for avatars in VR. Facially expressive avatars is a research prototype that allows users to reflect their real facial gestures on their virtual avatars in real‐time. For example, if you smile, your virtual avatar smiles as well. They conducted focus groups with VR developers about the potential value of facial expressiveness and found that expressing both positive emotions (e.g., happiness, excitement, and humor) and negative emotions (e.g., confusion, fear, disgust) are valuable for VR avatars. For instance, expressing fear or disgust facially while playing a zombie game would be more authentic and more immersive. In this gallery presentation you will have the opportunity to try an Oculus VR headset and demo an immersive experience. You will continue by creating your own avatar and edit how it looks. This demonstration will serve as an example of the current state of social VR experiences, and highlight how face‐mimicking can improve the users' virtual reality experiences by allowing them to express a wide variety of emotions and build agency within social VR experiences.

Progressive Training for Motor Imagery Brain-Computer Interfaces Using Gamification and Virtual Reality Embodiment.

This paper presents a gamified motor imagery brain-computer interface (MI-BCI) training in immersive virtual reality. The aim of the proposed training method is to increase engagement, attention, and motivation in co-adaptive event-driven MI-BCI training. This was achieved using gamification, progressive increase of the training pace, and virtual reality design reinforcing body ownership transfer (embodiment) into the avatar. From the 20 healthy participants performing 6 runs of 2-class MI-BCI training (left/right hand), 19 were trained for a basic level of MI-BCI operation, with average peak accuracy in the session = 75.84%. This confirms the proposed training method succeeded in improvement of the MI-BCI skills; moreover, participants were leaving the session in high positive affect. Although the performance was not directly correlated to the degree of embodiment, subjective magnitude of the body ownership transfer illusion correlated with the ability to modulate the sensorimotor rhythm.

Virtual Reality Avatar Therapy for People Hearing Voices

Virtual Reality Avatar Therapy for People Hearing Voices

Facial Identification at a Virtual Reality Airport.

Person identification at airports requires the comparison of a passport photograph with its bearer. In psychology, this process is typically studied with static pairs of face photographs that require identity-match (same person shown) versus mismatch (two different people) decisions, but this approach provides a limited proxy for studying how environment and social interaction factors affect this task. In this study, we explore the feasibility of virtual reality (VR) as a solution to this problem, by examining the identity matching of avatars in a VR airport. We show that facial photographs of real people can be rendered into VR avatars in a manner that preserves image and identity information (Experiments 1 to 3). We then show that identity matching of avatar pairs reflects similar cognitive processes to the matching of face photographs (Experiments 4 and 5). This pattern holds when avatar matching is assessed in a VR airport (Experiments 6 and 7). These findings demonstrate the feasibility of VR as a new method for investigating face matching in complex environments.