Video Conferencing in the Age of Covid-19: Engaging Online Interaction Using Facial Expression Recognition and Supplementary Haptic Cues

Abstract

More than 50 years since its mass market introduction the core user interfaces of Video Conferencing (VC) systems have essentially been unchanged. Relaying real time audio and video over distance is inherently productive. However, it lacks the sense of in-person interaction. With the current global pandemic, additional privacy concerns over the extended use of video and audio-conferencing systems, there is a need to redefine how VC Systems function and what information they communicate. To resolve these issues, we propose a VC system that utilizes facial recognition to identify and catalog participant’s expressions and communicates their emotional states to other participants on the VC system using encoded haptic cues. In our testing we found that the approach was able to provide summarized haptic feedback of facial expressions and reduce the time it took for the participants to react to ongoing discussions without increasing mental or physical strain on the user.