Virtual Avatar Research Articles

Exploring the acceptability, and feasibility of a modified virtual reality-based AVATAR therapy in schizophrenia spectrum disorders: A case series report

Schizophrenia spectrum disorder poses a complex challenge in psychiatric treatment due to its multifaceted symptomatology. Modified AVATAR therapy, an innovative virtual reality-based intervention integrating Cognitive Behavioral Therapy (CBT) techniques such as systematic desensitization, and Acceptance and Commitment Therapy (ACT) approach offers a promising avenue for addressing auditory verbal hallucinations (AVH). This case series report investigates the acceptability and feasibility of a modified AVATAR therapy in three patients diagnosed with schizophrenia spectrum disorder who experience distressing AVH. The study provides detailed demographic and clinical information, outlines the modified AVATAR therapy protocol, and presents outcomes measured through standardized assessments.The findings indicate a decrease in the severity of AVHs, along with enhancements in overall symptomatology after modified AVATAR therapy sessions. Moreover, qualitative perspectives from patients shed light on their favorable experiences and perceptions of this adapted intervention. Nonetheless, the results exhibited inconsistency across individual cases, underscoring challenges for future research and clinical applications in this domain. Nevertheless, in light of the hurdles accompanying the rehabilitation of schizophrenia patients, along with the evaluation and subsequent measurement of parameters affecting efficacy, modified AVATAR therapy could present a valuable addition to schizophrenia patient care.

Exploration of AI and AR Technologies in the Character Design of "Dream of the Red Chamber"

This study explores the potential of artificial intelligence (AI) and augmented reality (AR) technologies in recreating the character of Wang Xifeng, a core figure in the Chinese classical literary masterpiece "Dream of the Red Mansions". By integrating two advanced tools, Midjourney and Kivicube, the study successfully created and displayed a digital virtual avatar of Wang Xifeng. The research employed qualitative research methods, including content analysis, experimental research, and a questionnaire survey. The feedback results showed that most participants were satisfied with the reshaped image of Wang Xifeng using AI and AR technologies. This study not only opens up new avenues for the modern interpretation of traditional literary characters but also provides valuable experience and theoretical support for the inheritance and innovative practice of traditional culture.

Multi-Modal Emotional Understanding in AI Virtual Characters: Integrating Micro-Expression-Driven Feedback within Context-Aware Facial Micro-Expression Processing Systems

To engage users, AI Virtual Characters must comprehend emotions. The paper develops and evaluates Chinese-specific context-aware facial micro-expression processing algorithms and feedback mechanisms to improve AI virtual characters' multi-modal emotional comprehension in Chinese culture. Specialized algorithms were used to collect and evaluate Chinese micro-expressions and assess AI virtual characters' emotional comprehension in user interactions. Chinese participants of various ages, genders, and places were recruited for micro-expression recognition to ensure cultural inclusion. A comprehensive method collected quantitative and qualitative data. We integrated interview and AI virtual character feedback with quantitative indicators like emotion recognition accuracy, user engagement, and micro-expression intensity. In the study, demographics affect emotion recognition accuracy and age, gender, and location-specific virtual avatars increase emotional resonance. A study demonstrated that context influences micro-expression interpretation, particularly in distinguishing urban and rural surprise and grief. Textual micro-expression recommendations and real-time AI character expression modifications increased accuracy and user experience. Micro-expressions, visual and auditory cues, and physiological reactions are linked, requiring multimodal signal processing for emotional awareness. Interactive virtual help, gaming, and education may benefit from culturally appropriate AI characters. Deep learning, multimodal fusion, and explainable AI are used in AI emotional interaction theory and technique. Finally, using Chinese cultural intricacies, our study improves AI virtual character multi-modal emotional comprehension. Culturally sensitive AI personalities and emotional AI technologies are developed using context-aware face micro-expression analysis algorithms and feedback systems.

CLO 3D Customized Virtual Avatar Modeling and Prototype Development -Designing Size-adjustable Maternity Wear-

CLO 3D Customized Virtual Avatar Modeling and Prototype Development -Designing Size-adjustable Maternity Wear-

A randomized controlled trial investigating experiential virtual reality communication on prudent antibiotic use

Antimicrobial resistance (AMR) is a global health threat. This randomized controlled trial evaluates the impact of experiential virtual reality (VR) versus information provision via VR or leaflet on prudent antibiotic use. A total of 249 (239 analyzed) participants were randomized into three conditions: VR Information + Experience, VR Information, or Leaflet Information. All participants received AMR information, while those in the VR Information + Experience condition additionally engaged in a game, making treatment decisions for their virtual avatar’s infection. Participants in the VR Information + Experience condition showed a significant increase in prudent use intentions from baseline (d = 1.48). This increase was significantly larger compared to the VR Information (d = 0.50) and Leaflet Information (d = 0.79) conditions. The increase in intentions from baseline remained significant at follow-up in the VR Information + Experience condition (d = 1.25). Experiential VR communication shows promise for promoting prudent antibiotics use.

Exploring the Field of Virtual Avatar: A Bibliometric and Content Analysis

The virtual avatar (VA) concept has gained significant attention since the introduction of virtual products. This study aims to investigate the present level of advancement and forthcoming research directions in the domain of VA while also constructing a conceptual framework for VA. A thorough examination of the VA field was carried out by integrating bibliometric and content analytic techniques. Eight hundred ninety-nine journal articles underwent descriptive bibliometric analysis using CiteSpace and VOSviewer software to construct and visualize synthetic networks. Two hundred forty-eight documents were selected for content analysis. VA is mostly focused on a few specific disciplines and journals prioritizing innovation. Conducting empirical investigations in many settings can enhance the research areas of Virtual Reality (VR), Augmented Reality (AR), and Extended Reality (XR), including improving algorithms, exploring accessibility usage, and examining public service applications. Alternative theoretical frameworks can provide novel insights and additional directions for the development of the VA field.

Sense of embodiment with synchronized avatar during walking in mixed reality.

Gait guidance systems that synchronize the gait rhythm with an avatar in a mixed reality (MR) environment are attracting attention owing to their rehabilitation applications. More effective gait guidance can be achieved by changing body sensations for the sense of embodiment (SoE), which refers to the feeling of owning, controlling, and being inside a body in MR. This study investigated full-body synchronous motion between a human and a virtual avatar to enhance the SoE in walking with actual position changes in the real world. The full-body motion and gait rhythm were measured using body-worn inertial measurement units and a visual avatar was provided through a transparent head-mounted display. The results showed that the SoE of the participants was enhanced under higher synchronization conditions. In addition, questionnaire results showed that the SoE in the synchronous condition was significantly higher than that in the asynchronous condition, and the SoE in the self-avatar condition was significantly higher than that in the other-avatar condition. This indicates that a higher synchronization level with the appearance of an avatar leads to a stronger SoE in the human perception mechanism, which is important for potential application in medical or other fields.

Teaching the Teachers: Technological Proficiency and Professional Growth in Rural Education

Preparing teachers in small rural schools to effectively integrate technology into their teaching practices is vital for fostering 21st-century skills in students, despite the unique challenges these settings present. To explore the specific challenges and opportunities related to the professional development of teachers in small rural schools, in the context of increasing digitalisation of education. This study employs a systematic review as its research method, aiming to identify, critically evaluate, and synthesize relevant studies that shed light on this context. Teacher education programmes that adeptly incorporate technology, such as mobile devices, into their curricula offer a critical support system not just for urban educators, but importantly for rural teacher candidates who may have limited access to higher education resources. Smartphones and tablets, for instance, can serve as valuable tools for these teachers, allowing them constant and remote access to professional development materials and effective learning resources. The technology also presents an opportunity for iterative learning, where immediate feedback can guide improvements in teaching strategies. Examples include the use of virtual avatars as teaching aids instead of in-person instruction. As the push for digitalisation continues, it is incumbent upon educational institutions to articulate the benefits and returns of such investments, especially in the context of small rural schools where resources may be limited and the challenges distinct. This articulation must be framed in terms that resonate with educators, students, communities, and potential educational funders in these rural contexts.

Shaping corticospinal pathways in virtual reality: effects of task complexity and sensory feedback during mirror therapy in neurologically intact individuals

BackgroundRestoration of limb function for individuals with unilateral weakness typically requires volitional muscle control, which is often not present for individuals with severe impairment. Mirror therapy—interventions using a mirror box to reflect the less-impaired limb onto the more-impaired limb—can facilitate corticospinal excitability, leading to enhanced recovery in severely impaired clinical populations. However, the mirror box applies limitations on mirror therapy, namely that all movements appear bilateral and are confined to a small area, impeding integration of complex activities and multisensory feedback (e.g., visuo-tactile stimulation). These limitations can be addressed with virtual reality, but the resulting effect on corticospinal excitability is unclear.ObjectiveExamine how virtual reality-based unilateral mirroring, complex activities during mirroring, and visuo-tactile stimulation prior to mirroring affect corticospinal excitability.Materials and methodsParticipants with no known neurological conditions (n = 17) donned a virtual reality system (NeuRRoVR) that displayed a first-person perspective of a virtual avatar that matched their motions. Transcranial magnetic stimulation-induced motor evoked potentials in the nondominant hand muscles were used to evaluate corticospinal excitability in four conditions: resting, mirroring, mirroring with prior visuo-tactile stimulation (mirroring + TACT), and control. During mirroring, the movements of each participant’s dominant limb were reflected onto the nondominant limb of the virtual avatar, and the avatar’s dominant limb was kept immobile (i.e., unilateral mirroring). The mirroring + TACT condition was the same as the mirroring condition, except that mirroring was preceded by visuo-tactile stimulation of the nondominant limb. During the control condition, unilateral mirroring was disabled. During all conditions, participants performed simple (flex/extend fingers) and complex (stack virtual blocks) activities.ResultsWe found that unilateral mirroring increased corticospinal excitability compared to no mirroring (p < 0.001), complex activities increased excitability compared to simple activities during mirroring (p < 0.001), and visuo-tactile stimulation prior to mirroring decreased excitability (p = 0.032). We also found that these features did not interact with each other.DiscussionsThe findings of this study shed light onto the neurological mechanisms of mirror therapy and demonstrate the unique ways in which virtual reality can augment mirror therapy. The findings have important implications for rehabilitation for design of virtual reality systems for clinical populations.

Enhancing motor imagery in the third-person perspective by manipulating sense of body ownership with virtual reality.

Virtual reality (VR)-guided motor imagery (MI) is a widely used approach for motor rehabilitation, especially for patients with severe motor impairments. Most approaches provide visual guidance from the first-person perspective (1PP). MI training with visual guidance from the third-person perspective (3PP) remains largely unexplored. We argue that 3PP MI training has its own advantages and can supplement 1PP MI. For some movements beyond the view of 1PP, such as shoulder shrugging and other axial movements, MI are suitable performed under 3PP. However, the efficiency of existing paradigms for 3PP MI is unsatisfactory. We speculate that the absence of sense of body ownership (SOO) from 3PP could be one possible factor and hypothesize that 3PP MI could be enhanced by eliciting SOO over a 3PP avatar. Based on our hypothesis, a novel paradigm was proposed to enhance 3PP MI by inducing full-body illusion (FBI) from 3PP, which is similar to the so-called out-of-body experience (OBE), using synchronous visuo-tactile stimulus with VR. The event-related Electroencephalograph (EEG) desynchronization (ERD) at motor-related regions from 31 healthy participants were calculated and compared with a control paradigm without "OBE" FBI induction. This study attempts to enhance 3PP MI with FBI induction. It offers an opportunity to perform MI guided by action observation from 3PP with elicited SOO to the observed avatar. We believe that 3PP MI could provide more possibilities for effective rehabilitation training, when SOO could be elicited to a virtual avatar and the present work demonstrates its viability and effectiveness.

Tele-Impedance control of a virtual avatar based on EMG and M-IMU sensors: a proof-of-concept

The broad spread of cooperative robots into many application domains has resulted in a demand for intuitive and effective solutions for teleoperated control. A relevant role in teleoperation has been assumed by impedance controllers, that allow the increase of stability and accuracy during interaction. This paper aims to test a teleoperation method based on an impedance controller, namely tele-impedance control, that is usable in unstructured environments since it relies only on wearable sensors. The proposed solution maps the joint stiffness and position of the human user, computed through six EMG and two M-IMU sensors, into the remote system to be teleoperated. We developed a 2-DoFs virtual task involving virtual physical interactions to compare the performance of our solution with the one of a traditional position-based controller. The study has been conducted on five healthy participants, who experienced both controllers in two different sessions. The tele-impedance approach has proved to be less physically demanding and more intuitive than the position-based one. Experimental data also allow us to investigate the strategy employed by the volunteers in the case of remote interactions, while using the two controllers. Of note, even though with the position controller the variation of subject impedance has no effect on the virtual arm, participants still tend to regulate both impedance and position of their own arm.

DialogueNeRF: towards realistic avatar face-to-face conversation video generation

Conversation is an essential component of virtual avatar activities in the metaverse. With the development of natural language processing, significant breakthroughs have been made in text and voice conversation generation. However, face-to-face conversations account for the vast majority of daily conversations, while most existing methods focused on single-person talking head generation. In this work, we take a step further and consider generating realistic face-to-face conversation videos. Conversation generation is more challenging than single-person talking head generation, because it requires not only the generation of photo-realistic individual talking heads, but also the listener’s response to the speaker. In this paper, we propose a novel unified framework based on the neural radiance field (NeRF) to address these challenges. Specifically, we model both the speaker and the listener with a NeRF framework under different conditions to control individual expressions. The speaker is driven by the audio signal, while the response of the listener depends on both visual and acoustic information. In this way, face-to-face conversation videos are generated between human avatars, with all the interlocutors modeled within the same network. Moreover, to facilitate future research on this task, we also collected a new human conversation dataset containing 34 video clips. Quantitative and qualitative experiments evaluate our method in different aspects, e.g., image quality, pose sequence trend, and natural rendering of the scene in the generated videos. Experimental results demonstrate that the avatars in the resulting videos are able to carry on a realistic conversation, and maintain individual styles.

Creating a metaverse‐me: Exploring the consumer avatar creation process

AbstractThe metaverse is emerging as a focal point for technological advancement, social interaction, and commerce. Despite brands' enthusiasm for establishing a presence in the metaverse, consumer adoption remains modest, with the initial hurdle being avatar creation. To better understand why consumers are hesitant to engage in the metaverse, this study focuses on avatars as virtual representations of self, particularly during the stage of avatar creation. Drawing on theories of self‐presentation and identity formation, this research investigates how users conceptualise and express their virtual avatars. Through a series of in‐depth semi‐structured interviews, we propose an integrated framework that illustrates how consumers engage in a complex avatar creation process consisting of consolidation, exploration, affirmation, and aspiration. Further, our insights show that avatars serve as extensions of real‐world identities, as well as how this impacts consumption behaviour. These insights are valuable for marketers aiming to develop actionable consumer‐centric metaverse strategies, and researchers investigating consumer behaviour within immersive consumption environments.

Augmented-reality-based multi-person exercise has more beneficial effects on mood state and oxytocin secretion than standard solitary exercise

BackgroundExercise has positive effects on psychological well-being, with team sports often associated with superior mental health compared to individual sports. Augmented reality (AR) technology has the potential to convert solitary exercise into multi-person exercise. Given the role of oxytocin in mediating the psychological benefits of exercise and sports, this study aimed to investigate the impact of AR-based multi-person exercise on mood and salivary oxytocin levels. MethodsFourteen participants underwent three distinct regimens: non-exercise (Rest), standard solitary cycling exercise (Ex), and AR-based multi-person cycling exercise (Ex+AR). In both Ex and Ex+AR conditions, participants engaged in cycling at a self-regulated pace to maintain a Rating of Perceived Exertion of 10. In the Ex+AR condition, participants' avatars were projected onto a tablet screen, allowing them to cycle alongside ten other virtual avatars in an AR environment. Mood states and saliva samples were collected before and immediately after each 10-minute regimen. Subsequently, salivary oxytocin levels were measured. ResultsNotably, only the Ex+AR condition significantly improved mood states associated with depression-dejection and exhibited a non-significant trend toward suppressing anger-hostility in participants. Moreover, the Ex+AR condition led to a significant elevation in salivary oxytocin levels, while the Ex condition showed a non-significant trend toward an increase. However, changes in salivary oxytocin did not show a significant correlation with changes in mood states. ConclusionsThese findings suggest that Ex+AR enhances mood states and promotes oxytocin release. AR-based multi-person exercise may offer greater psychological benefits compared to standard solitary exercise, although the relationship between oxytocin and mood changes remains inconclusive.

Action unit intensity regression for facial MoCap aimed towards digital humans

AbstractDue to the increasing demand for virtual avatars, there has been a recent growth in the research and development of frameworks for realistic digital humans, which create a demand for realistic and adaptable facial motion capture systems. Most frameworks belong to private companies or represent high investments, which is why the creation of democratized solutions is relevant for the growth of digital human content creation. This research work proposes a facial motion capture framework for digital humans with the use of machine learning for facial codification intensity regression. The main focus is to use coded face movement intensities to generate realistic expressions on a digital human. The ablation studies performed on the regression models show that Neural Networks, using Histogram of Oriented Gradients as features, and with person-specific normalization, present overall better performance against other methods in the literature. With an RMSE of 0.052, the proposed framework offers reliable results that can be rendered in the face of a MetaHuman.

A new XR-based human‐robot collaboration assembly system based on industrial metaverse

The industrial metaverse is an emerging paradigm of integrated information technologies. A human-robot collaboration assembly system is developed using extended reality (XR) and blockchain technologies. This system includes the physical, virtual reality (VR), augmented reality (AR) supporting, and blockchain systems. Virtual avatars represent physical operators and mobile robots in the XR environment. The VR system emulates real-time physical operations, facilitating pre-planning, online task optimization, and converting strategic decisions into executable instructions for physical robots. The AR system enhances the cognitive experience of intelligent operators, contributing to decision-making and VR system control through multi-modal interactions. The integration of the VR and AR systems in perception-decision-control enables real-time process optimization and the reverse control of physical operations. The system is implemented using a decentralized blockchain and end-edge-cloud collaborative network architecture to regulate interaction permissions for each scene characters, ensuring systematic functionality, robust communication, and comprehensive data management. A case study of a gearbox assembly is used to demonstrate the system’s development and efficacy. The results indicate the VR system's comprehensive perception and the AR system's detailed insights, yielding reliable information on global and local levels. The integration of artificial intelligence (AI) algorithms and spatiotemporal visualization during VR decision-making provides computational advantages. Incorporating human input into the AR system enables human-in-the-loop operational decision-making, providing optimal instructions to improve human-robot collaboration performance. Notably, the fusion of blockchain and end-edge-cloud architectures augments inter-system data exchange by efficiently managing and processing human-robot interactions and their diverse avatars. This system offers an innovative approach to human-robot collaboration assembly in manufacturing, with implications for analogous contexts.

Auswirkungen der visuellen Flussgeschwindigkeit auf das Radfahrerlebnis in der virtuellen Realität

AbstractApplying virtual reality to exercise has revealed mood- and performance-enhancing properties of immersive media. Social–cognitive theory and the Proteus effect suggest that avatar appearance contributes to this relation by eliciting behavioral changes. Attempting to influence exercise parameters without modifying the virtual avatar, the present study investigated the effects of differing visual flow speeds on physiological and perceived effort during aerobic exercise. Eighty-two university students participated in three separate experimental trials. During each trial, a virtual cycling track was presented at one of three velocities (16, 20, 24 km/h) in counterbalanced order, while participants cycled at a moderate intensity for 20 min. Objective and subjective measures of effort and affective states were recorded every five minutes. With increasing visual flow speed, a linear decrease of heart rate, perceived effort, and arousal and a linear increase of valence were expected. Mixed linear model analyses revealed no significant main effect of visual flow speed on any dependent variable. A nonlinear relation between visual flow speed and heart rate was identified through pairwise comparisons between visual flow conditions.

Developmental changes of bodily self-consciousness in adolescent girls

The body and the self change markedly during adolescence, but how does bodily self-consciousness, the pre-reflexive experience of being a bodily subject, change? We addressed this issue by studying embodiment towards virtual avatars in 70 girls aged 10–17 years. We manipulated the synchrony between participants’ and avatars’ touch or movement, as well as the avatar visual shape or size relative to each participant’s body. A weaker avatar’s embodiment in case of mismatch between the body seen in virtual reality and the real body is indicative of a more robust bodily self-consciousness. In both the visuo-tactile and the visuo-motor experiments, asynchrony decreased ownership feeling to the same extent for all participants, while the effect of asynchrony on agency feeling increased with age. In the visuo-tactile experiment, incongruence in visual appearance did not affect agency feeling but impacted ownership, especially in older teenage girls. These findings highlight the higher malleability of bodily self-consciousness at the beginning of adolescence and suggest some independence between body ownership and agency.

One-Shot Face Stylization

The abstract delves into the intricate realm of one-shot face stylization, a captivating domain within computer vision and deep learning. It revolves around the art of transforming a target face using a reference image, navigating the delicate balance between retaining facial recognition and infusing desired stylistic traits. This pursuit has garnered considerable attention owing to its myriad applications across digital art, entertainment, and personalized products. The abstract scrutinizes the essential components of one-shot face stylization, highlighting the pivotal role of deep neural networks, particularly generative adversarial networks (GANs). These networks are adept at crafting bespoke facial images by assimilating information from both the target and reference faces, leveraging the reference image as a guiding beacon. The crux of success in one-shot facial stylization lies in the meticulous orchestration of the fading process, which harmonizes the preservation of identity with the enhancement of artistic technique. As advancements in this field continue to unfold, the potential ramifications span far and wide, promising to revolutionize creative expression and customization across diverse industries, spanning from digital art and animation to virtual avatars and social media filters. Key Words: One-Shot face stylization, Generative adversarial network, Facial recognition, Digital art, Personalization

Spatiotemporal Inconsistency Learning and Interactive Fusion for Deepfake Video Detection

With the rise of the metaverse, the rapid advancement of Deepfakes technology has become closely intertwined. Within the metaverse, individuals exist in digital form and engage in interactions, transactions, and communications through virtual avatars. However, the development of Deepfakes technology has led to the proliferation of forged information disseminated under the guise of users’ virtual identities, posing significant security risks to the metaverse. Hence, there is an urgent need to research and develop more robust methods for detecting deep forgeries to address these challenges. This paper explores deepfake video detection by leveraging the spatiotemporal inconsistencies generated by deepfake generation techniques, and thereby proposing the interactive spatioTemporal inconsistency learning and interactive fusion (ST-ILIF) detection method, which consists of phase-aware and sequence streams. The spatial inconsistencies exhibited in frames of deepfake videos are primarily attributed to variations in the structural information contained within the phase component of the Fourier domain. To mitigate the issue of overfitting the content information, a phase-aware stream is introduced to learn the spatial inconsistencies from the phase-based reconstructed frames. Additionally, considering that deepfake videos are generated frame-by-frame and lack temporal consistency between frames, a sequence stream is proposed to extract temporal inconsistency features from the spatiotemporal difference information between consecutive frames. Finally, through feature interaction and fusion of the two streams, the representation ability of intermediate and classification features is further enhanced. The proposed method, which was evaluated on four mainstream datasets, outperformed most existing methods, and extensive experimental results demonstrated its effectiveness in identifying deepfake videos. Our source code is available at https://github.com/qff98/Deepfake-Video-Detection