Abstract
Recent developments of social virtual reality (VR) services using avatars have increased the need for facial expression recognition (FER) technology. FER systems are generally implemented using optical cameras; however, the performance of these systems can be limited when users are wearing head-mounted displays (HMDs) as users' faces are largely covered by the HMDs. Facial electromyograms (fEMGs) that can be recorded around users' eyes can be potentially used for implementing FER systems for VR applications. However, this technology lacks practicality owing to the need for large-scale training datasets; furthermore, it is hampered by a relatively low performance. In this study, we proposed an fEMG-based FER system based on the Riemannian manifold-based approach to reduce the number of training datasets needed and enhance FER performance. Our experiments with 42 participants showed an average classification accuracy as high as 85.01% for recognizing 11 facial expressions with only a single training dataset for each expression. We further developed an online FER system that could animate a virtual avatar's expression reflecting a user's facial expression in real time, thus demonstrating that our FER system can be potentially used for practical interactive VR applications, such as social VR networks, smart education, and virtual training.
Highlights
With rapid developments in virtual reality (VR) technologies, there is an increasing interest in social network VR applications using human or human-like avatars [1]–[4]
RIEMANNIAN MANIFOLD-BASED APPROACH Recently, Riemannian manifold-based pattern classification has attracted much interest for brain computer interface (BCI) applications as it has been reported that BCI performance can be significantly improved using this approach [28]–[30]
To implement an online Facial expression recognition (FER) system with a speed of 20 frames per second, the classification decision should be made within 50 ms
Summary
With rapid developments in virtual reality (VR) technologies, there is an increasing interest in social network VR applications using human or human-like avatars [1]–[4]. Facebook recently released a new VR social network application called Facebook Space [5] and started to evolve their mobile-based social network service (SNS) platform into a VR-based SNS. Various VR applications, such as Vtime, VRchat, AltSpaceVR, and High Fidelity VR have been released into the market to keep pace with rapidly changing social interaction environments [6], [7]. To allow users a more immersive experience during VR-based social interaction, it is important to precisely recognize their facial expressions and visualize them on an avatar in VR. Facial expression recognition (FER) has been studied widely using optical cameras in the field of computer vision [8]–[13].
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
