Avatar Representation Research Articles

Avatars and the Value of Human Uniqueness

Danaher and Nyholm (Philosophy & Technology 37:106, 2024) explore whether avatar technology makes humans less valuable by making them less scarce. They identify two forms of human scarcity, intrinstic scarcity and instrumental scarcity, and explore how each is impacted by avatar representation. Here I argue that avatars cannot make humans less scarce but that, nevertheless, the use of avatar technology can undermine the value of human uniqueness.

Virtual reality in teaching using mentally ill patient avatars

Medical interaction and exploration techniques are the most important tools that medical students have to acquire in the subject of psychiatry and psychotherapy. The new digital technologies currently available, such as virtual reality (VR), as important supplements can contribute to asignificant improvement in the teaching of psychiatric-psychopathological learning content as well as, in particular, the technique of ascertaining the psychiatric history and diagnosis. Evaluation of the Bochum Avatar Exploration Project (AVEX) as part of the curricular course in medical studies at the Ruhr University Bochum for its possibilities to convey learning content and techniques of anamnesis and diagnosis in the subject of psychiatry and psychotherapy. In AVEX, atotal of 87medical students in the clinical study section have so far been able to enter into a dialogue with "mentally ill" avatars and gain experience with VR technology as alearning and teaching method in the subject of psychiatry and psychotherapy. Despite the limited possibilities for interaction with the digital avatars, it is possible to achieve asubstantial transfer of learning content in psychiatry; however, the students must be well supported by the lecturers. The AVEX project already shows promising possibilities for supplementing the teaching of medical students, even if the fit of questions and replies in dialogue with the virtual avatars still needs to be improved. As advances in the linguistic communication of emotions and the visual effects of the avatar representation can be predicted, the significance of this technology will continue to increase.

Investigating the Effects of Avatarization and Interaction Techniques on Near-field Mixed Reality Interactions with Physical Components.

Mixed reality (MR) interactions feature users interacting with a combination of virtual and physical components. Inspired by research investigating aspects associated with near-field interactions in augmented and virtual reality (AR & VR), we investigated how avatarization, the physicality of the interacting components, and the interaction technique used to manipulate a virtual object affected performance and perceptions of user experience in a mixed reality fundamentals of laparoscopic peg-transfer task wherein users had to transfer a virtual ring from one peg to another for a number of trials. We employed a 3 (Physicality of pegs) X 3 (Augmented Avatar Representation) X 2 (Interaction Technique) multi-factorial design, manipulating the physicality of the pegs as a between-subjects factor, the type of augmented self-avatar representation, and the type of interaction technique used for object-manipulation as within-subjects factors. Results indicated that users were significantly more accurate when the pegs were virtual rather than physical because of the increased salience of the task-relevant visual information. From an avatar perspective, providing users with a reach envelope-extending representation, though useful, was found to worsen performance, while co-located avatarization significantly improved performance. Choosing an interaction technique to manipulate objects depends on whether accuracy or efficiency is a priority. Finally, the relationship between the avatar representation and interaction technique dictates just how usable mixed reality interactions are deemed to be.

Effects of Avatar Transparency on Social Presence in Task-centric Mixed Reality Remote Collaboration.

Despite the importance of avatar representation on user experience for Mixed Reality (MR) remote collaboration involving various device environments and large amounts of task-related information, studies on how controlling visual parameters for avatars can benefit users in such situations have been scarce. Thus, we conducted a user study comparing the effects of three avatars with different transparency levels (Nontransparent, Semi-transparent, and Near-transparent) on social presence for users in Augmented Reality (AR) and Virtual Reality (VR) during task-centric MR remote collaboration. Results show that avatars with a strong visual presence are not required in situations where accomplishing the collaborative task is prioritized over social interaction. However, AR users preferred more vivid avatars than VR users. Based on our findings, we suggest guidelines on how different levels of avatar transparency should be applied based on the context of the task and device type for MR remote collaboration.

AvatarReX: Real-time Expressive Full-body Avatars

We present AvatarReX, a new method for learning NeRF-based full-body avatars from video data. The learnt avatar not only provides expressive control of the body, hands and the face together, but also supports real-time animation and rendering. To this end, we propose a compositional avatar representation, where the body, hands and the face are separately modeled in a way that the structural prior from parametric mesh templates is properly utilized without compromising representation flexibility. Furthermore, we disentangle the geometry and appearance for each part. With these technical designs, we propose a dedicated deferred rendering pipeline, which can be executed at a real-time framerate to synthesize high-quality free-view images. The disentanglement of geometry and appearance also allows us to design a two-pass training strategy that combines volume rendering and surface rendering for network training. In this way, patch-level supervision can be applied to force the network to learn sharp appearance details on the basis of geometry estimation. Overall, our method enables automatic construction of expressive full-body avatars with real-time rendering capability, and can generate photo-realistic images with dynamic details for novel body motions and facial expressions.

The assessment of presence and performance in an AR environment for motor imitation learning: A case-study on violinists

The acquisition of advanced gestures is a challenge in various domains of proficient sensorimotor performance. For example, orchestral violinists must move in sync with the lead violinist's gestures. To help train these gestures, an educational music play-back system was developed using a HoloLens 2 simulated AR environment and an avatar representation of the lead violinist. This study aimed to investigate the impact of using a 2D or 3D representation of the lead violinist's avatar on students' learning experience in the AR environment.To assess the learning outcome, the study employed a longitudinal experiment design, in which eleven participants practiced two pieces of music in four trials, evenly spaced over a month. Participants were asked to mimic the avatar's gestures as closely as possible when it came to using the bow, including bowing, articulations, and dynamics. The study compared the similarities between the avatar's gestures and those of the participants at the biomechanical level, using motion capture measurements, as well as the smoothness of the participants' movements. Additionally, presence and perceived difficulty were assessed using questionnaires.The results suggest that using a 3D representation of the avatar leads to better gesture resemblance and a higher experience of presence compared to a 2D representation. The 2D representation, however, showed a learning effect, but this was not observed in the 3D condition. The findings suggest that the 3D condition benefits from stereoscopic information that enhances spatial cognition, making it more effective in relation to sensorimotor performance. Overall, the 3D condition had a greater impact on performance than on learning.This work concludes with recommendations for future efforts directed towards AR-based advanced gesture training to address the challenges related to measurement methodology and participants' feedback on the AR application.

NeuroSuitUp: System Architecture and Validation of a Motor Rehabilitation Wearable Robotics and Serious Game Platform

This article presents the system architecture and validation of the NeuroSuitUp body-machine interface (BMI). The platform consists of wearable robotics jacket and gloves in combination with a serious game application for self-paced neurorehabilitation in spinal cord injury and chronic stroke. The wearable robotics implement a sensor layer, to approximate kinematic chain segment orientation, and an actuation layer. Sensors consist of commercial magnetic, angular rate and gravity (MARG), surface electromyography (sEMG), and flex sensors, while actuation is achieved through electrical muscle stimulation (EMS) and pneumatic actuators. On-board electronics connect to a Robot Operating System environment-based parser/controller and to a Unity-based live avatar representation game. BMI subsystems validation was performed using exercises through a Stereoscopic camera Computer Vision approach for the jacket and through multiple grip activities for the glove. Ten healthy subjects participated in system validation trials, performing three arm and three hand exercises (each 10 motor task trials) and completing user experience questionnaires. Acceptable correlation was observed in 23/30 arm exercises performed with the jacket. No significant differences in glove sensor data during actuation state were observed. No difficulty to use, discomfort, or negative robotics perception were reported. Subsequent design improvements will implement additional absolute orientation sensors, MARG/EMG based biofeedback to the game, improved immersion through Augmented Reality and improvements towards system robustness.

휴먼-메타버스 인터랙션의 기회와 발전방향

In the COVID-19 pandemic era, non-contact services were demanded and the use of extended reality and metaverse services increased rapidly in various applications. In this paper, we analyze Gather.town, ifland, Roblox, and ZEPETO metaverse platforms in terms of user interaction, avatar-based interaction, and virtual world authoring. Especially, we distinguish interactions among user input techniques that occur in the real world, avatar representation techniques to represent users in the virtual world, and interaction types that create a virtual world through user participation. Based on this work, we highlight the current trends and needs of human-metaverse interaction and forecast future opportunities and research directions.

Reaching articular limits can negatively impact embodiment in virtual reality.

Providing Virtual Reality(VR) users with a 3D representation of their body complements the experience of immersion and presence in the virtual world with the experience of being physically located and more personally involved. A full-body avatar representation is known to induce a Sense of Embodiment (SoE) for this virtual body, which is associated with improvements in task performance, motivation and motor learning. Recent experimental research on embodiment provides useful guidelines, indicating the extent of discrepancy tolerated by users and, conversely, the limits and disruptive events that lead to a break in embodiment (BiE). Based on previous works on the limit of agency under movement distortion, this paper describes, studies and analyses the impact of a very common yet overlooked embodiment limitation linked to articular limits when performing a reaching movement. We demonstrate that perceiving the articular limit when fully extending the arm provides users with an additional internal proprioceptive feedback which, if not matched in the avatar's movement, leads to the disruptive realization of an incorrect posture mapping. This study complements previous works on self-contact and visuo-haptic conflicts and emphasizes the risk of disrupting the SoE when distorting users' movements or using a poorly-calibrated avatar.

The Influence of Avatar Representation on Interaction Force in Virtual Environment

The Influence of Avatar Representation on Interaction Force in Virtual Environment

Towards Balancing VR Immersion and Bystander Awareness

Head-mounted displays (HMDs) increase immersion into virtual worlds. The problem is that this limits headset users' awareness of bystanders: headset users cannot attend to bystanders' presence and activities. We call this the HMD boundary. We explore how to make the HMD boundary permeable by comparing different ways of providing informal awareness cues to the headset user about bystanders. We adapted and implemented three visualization techniques (Avatar View, Radar and Presence++) that share bystanders' location and orientation with headset users. We conducted a hybrid user and simulation study with three different types of VR content (high, medium, low interactivity) with twenty participants to compare how these visualization techniques allow people to maintain an awareness of bystanders, and how they affect immersion (compared to a baseline condition). Our study reveals that a see-through avatar representation of bystanders was effective, but led to slightly reduced immersion in the VR content. Based on our findings, we discuss how future awareness visualization techniques can be designed to mitigate the reduction of immersion for the headset user.

Virtual reality perspective-taking at scale: Effect of avatar representation, choice, and head movement on prosocial behaviors

The present investigation examined the effect of avatar representation, choice, and head movement on prosocial behaviors and measures of presence after a virtual reality perspective-taking (VRPT) task. Participants were either represented by a set of virtual hands or had no representation during a VRPT task. Of those with hands, only half were able to choose their skin tone. Results showed that there was no significant advantage to having an avatar representation. However, if participants had an avatar and were able to choose their own skin tone, a higher proportion of participants performed prosocial behaviors and reported higher social presence scores compared with participants who had no choice. Regardless of condition, head rotations significantly predicted petition signatures such that the more participants rotated their heads side to side, the more likely they were to sign the petition. Moreover, when participants do not consistently rotate their head side to side, the proportion of petitions signed is on par with individuals who do not complete a VRPT task at all.

Buddy biking: a user study on social collaboration in a virtual reality exergame for rehabilitation

Virtual reality (VR)-based rehabilitation is a growing technological field, which gradually becomes integrated into existing programs. However, technology has to support human behavior and -needs, including social relatedness, to achieve health-related outcomes. Elderly people have high risk of loneliness, and VR has technological affinity for natural social interaction. Previous studies have relied on competitiveness rather than collaborative elements, but research shows that competitiveness can lead to (feelings of) stress and aggressive behavior in some individuals. This article presents a mixed methods study to gather end-user feedback on a social VR scenario that encourages inter-player collaboration on a virtual tandem bike. Outpatients (n=11, 64% males, 60pm 11 years) were invited to participate with a co-player (friend or family). Participants biked on average 10.7 (± 3) minutes with a mean speed of 14.8 kmph (± 5.8). The results indicate potential and feasibility for the collaborative social biking application. Participants reported excellent usability-scores (85 ± 5), high intrinsic motivation in all categories: enjoyment (6.5 ± 0.5), effort/importance (6.4 ± 0.3), relatedness (6.3 ± 0.7) and minimal increase in symptoms of nausea, oculomotor and disorientation. Furthermore, participants found the social aspect enjoyable, agreed that collaboration eased tasks and that they lost track of exercise duration. Interpersonal interaction between participants varied, but was mostly positively rated valence, even if the sense of copresence was limited by physical constraints and avatar representation. Most participants expressed that they would use the program again, but future studies should explore how to improve location and appearance of the virtual coactor, as well as implement additional tasks.

The Influence of Avatar Representation on Interpersonal Communication in Virtual Social Environments.

Current avatar representations used in immersive VR applications lack features that may be important for supporting natural behaviors and effective communication among individuals. This study investigates the impact of the visual and nonverbal cues afforded by three different types of avatar representations in the context of several cooperative tasks. The avatar types we compared are No_Avatar (HMD and controllers only), Scanned_Avatar (wearing an HMD), and Heal_Avatar (video-see-through). The subjective and objective measures we used to assess the quality of interpersonal communication include surveys of social presence, interpersonal trust, communication satisfaction, and attention to behavioral cues, plus two behavioral measures: duration of mutual gaze and number of unique words spoken. We found that participants reported higher levels of trustworthiness in the Real_Avatar condition compared to the Scanned_Avatar and No_Avatar conditions. They also reported a greater level of attentional focus on facial expressions compared to the No_Avatar condition and spent more extended time, for some tasks, attempting to engage in mutual gaze behavior compared to the Scanned_Avatar and No_Avatar conditions. In both the Heal_Avatar and Scanned_Avatar conditions, participants reported higher levels of co-presence compared with the No_Avatar condition. In the Scanned_Avatar condition, compared with the Heal_Avatar and No_Avatar conditions, participants reported higher levels of attention to body posture. Overall, our exit survey revealed that a majority of participants (66.67%) reported a preference for the Real_Avatar, compared with 25.00% for the Scanned_Avatar and 8.33% for the No_Avatar, These findings provide novel insight into how a user's experience in a social VR scenario is affected by the type of avatar representation provided.

Trick the body trick the mind: avatar representation affects the perception of available action possibilities in virtual reality

In immersive Virtual Reality (VR), your brain can trick you into believing that your virtual hands are your real hands. Manipulating the representation of the body, namely the avatar, is a potentially powerful tool for the design of innovative interactive systems in VR. In this study, we investigated interactive behavior in VR by using the methods of experimental psychology. Objects with handles are known to potentiate the afforded action. Participants tend to respond faster when the handle is on the same side as the responding hand in bi-manual speed response tasks. In the first experiment, we successfully replicated this affordance effect in a Virtual Reality (VR) setting. In the second experiment, we showed that the affordance effect was influenced by the avatar, which was manipulated by two different hand types: (1) hand models with full finger tracking that are able to grasp objects, and (2) capsule-shaped—fingerless—hand models that are not able to grasp objects. We found that less than 5 mins of adaptation to an avatar, significantly altered the affordance perception. Counter intuitively, action planning was significantly shorter with the hand model that is not able to grasp. Possibly, fewer action possibilities provided an advantage in processing time. The presence of a handle speeded up the initiation of the hand movement but slowed down the action completion because of ongoing action planning. The results were examined from a multidisciplinary perspective and the design implications for VR applications were discussed.

Point-cloud avatars to improve spatial communication in immersive collaborative virtual environments

Collaborative virtual environments allow remote users to work together in a shared 3D space. To take advantage of the possibilities offered by such systems, their design must allow the users to interact and communicate efficiently. One open question in this field concerns the avatar fidelity of remote partners. This can impact communication between the remote users, more particularly when performing collaborative spatial tasks. In this paper, we present an experimental study comparing the effects of two partner’s avatars on collaboration during spatial tasks. The first avatar was based on a 2.5D streamed point-cloud and the second avatar was based on a 3D preconstructed avatar replicating the remote user movements. These avatars differ in their fidelity levels described through two components: visual and kinematic fidelity. The collaborative performance was evaluated through the efficacy of completing two spatial communication tasks, a pointing task and spatial guidance task. The results indicate that the streamed point-cloud avatar permitted a significant improvement of the collaborative performance for both tasks. The subjective evaluation suggests that these differences in performance can mainly be attributed to the higher kinematic fidelity of this representation as compared with the 3D preconstructed avatar representation. We conclude that, when designing spatial collaborative virtual environments, it is important to reach a high kinematic fidelity of the partner’s representation while a moderate visual fidelity of this representation can suffice.

Stereotype Threat in Virtual Learning Environments: Effects of Avatar Gender and Sexist Behavior on Women's Math Learning Outcomes.

Women in math, science, and engineering (MSE) often face stereotype threat: they fear that their performance in MSE will confirm an existing negative stereotype-that women are bad at math-which in turn may impair their learning and performance in math. This research investigated if sexist nonverbal behavior of a male instructor could activate stereotype threat among women in a virtual classroom. In addition, the research examined if learners' avatar representation in virtual reality altered this nonverbal process. Specifically, a 2 (avatar gender: female vs. male) × 2 (instructor behavior: dominant sexist vs. nondominant or nonsexist) between-subjects experiment was used. Data from 76 female college students demonstrated that participants learned less and performed worse when interacting with a sexist male instructor compared with a nonsexist instructor in a virtual classroom. Participants learned and performed equally well when represented by female and male avatars. Our findings extend previous research in physical learning settings, suggesting that dominant-sexist behaviors may give rise to stereotype threat and undermine women's learning outcomes in virtual classrooms. Implications for gender achievement gaps and stereotype threat are discussed.

Users’ psychophysiological, vocal, and self-reported responses to the apparent attitude of a virtual audience in stereoscopic 360°-video

This research analyzes the psychological reactions of participants to the neutral, positive, or negative attitudes of a virtual audience recorded in 360°-video. Participants were asked to deliver three speeches, each accompanied by a different type of reaction from the virtual audience. Measures of user state included questionnaires, psychophysiological measures, and voice recordings. The results showed that, compared to the neutral audience, the negative audience elicited increases in skin conductance level and heart rate variability, decreases in voice intensity, and a higher ratio of silent parts in the speech, as well as a more negative self-reported valence, higher anxiety, and lower social presence. These findings evidence that, even if some attributes that are considered to be central to immersive experiences are lacking, (i.e., interactivity or avatar representation of the user), 360°-video recreations of virtual environments lead to realistic reactions on users. These results support the effectiveness of 360°-video virtual audiences for public speaking training and social anxiety treatment.

Real-Time Avatar Pose Transfer and Motion Generation Using Locally Encoded Laplacian Offsets

We propose a human avatar representation scheme based on intrinsic coordinates, which are invariant to isometry and insensitive to human pose changes, and an efficient pose transfer algorithm that can utilize this representation to reconstruct a human body geometry following a given pose. Such a pose transfer algorithm can be used to control the movement of an avatar model in virtual reality environments following a user’s motion in real time. Our proposed algorithm consists of three main steps. First, we recognize the user’s pose and select a template model from the database who has a similar pose; then, the intrinsic Laplacian offsets encoded in local coordinates are used to reconstruct the human body geometry following the template pose; finally, the morphing between the two poses is generated using a linear interpolation. We perform experiments to evaluate the accuracy and efficiency of our algorithm. We believe our proposed system is a promising human modeling tool that can be used in general virtual reality applications.

User perceptions of avatar-based patient monitoring: a mixed qualitative and quantitative study

BackgroundA new patient monitoring technology called Visual Patient, which transforms numerical and waveform data into a virtual model (an avatar) of the monitored patient, has been shown to improve the perception of vital signs compared to conventional patient monitoring. In order to gain a deeper understanding of the opinions of potential future users regarding the new technology, we have analyzed the answers of two large groups of anesthetists using two different study methods.MethodsFirst, we carried out a qualitative analysis guided by the “consolidated criteria for reporting qualitative research” checklist. For this analysis, we interviewed 128 anesthesiologists, asking: “Where do you see advantages in Visual Patient monitoring?” and afterward identified major and minor themes in their answers. In a second study, an online survey with 38 anesthesiologists at two different institutions, we added a quantitative part in which anesthesiologists rated statements based on the themes identified in the prior analysis on an ordinal rating scale.ResultsWe identified four high-level themes: “quick situation recognition,” “intuitiveness,” “unique design characteristics,” and “potential future uses,” and eight subthemes.The quantitative questions raised for each major theme were: 1. “The Visual Patient technology enabled me to get a quick overview of the situation.” (63% of the participants agreed or very much agreed to this statement). 2. “I found the Visual Patient technology to be intuitive and easy to learn.” (82% agreed or very much agreed to this statement). 3. “The visual design features of the Visual Patient technology (e.g., the avatar representation) are not helpful for patient monitoring.” (11% agreed to this statement). 4. “I think the Visual Patient technology might be helpful for non-monitor experts (e.g., surgeons) in the healthcare system.” (53% of the participants agreed or strongly agreed).ConclusionThis mixed method study provides evidence that the included anesthesiologists considered the new avatar-based technology to be intuitive and easy to learn and that the technology enabled them to get an overview of the situation quickly. Only a few users considered the avatar presentation to be unhelpful for patient monitoring and about half think it might be useful for non-experts.