Emotion estimation of people wearing masks using machine learning

Abstract

Coronavirus (COVID-19) is the infectious agent responsible for the transmission of SARS (severe acute respiratory syndrome). When an infected person coughs, talks, or breathes, it spreads as small droplets of fluid from the mouth and nose of the infected person. Sanitary masks help prevent the spread of the virus from the person wearing the mask to others. This new behavior may cause a number of problems in interpersonal interactions. The goal of this paper is the emotion estimation of masked faces. It presents two major parts. At the first level, we created a system for identifying sanitary masks, using CNN. At the second one, we developed an emotion estimation system in order to estimate the classification rates of a masked face. It consists of three steps: face element detection, feature point localization, and classification. We used the well-known Viola and Jones algorithm in order to achieve the first step. We used several techniques to estimate emotions (SVM, KNN and deep learning). We made comparisons of the obtained results. Particular attention is also given to the effect of face masks on the performance of various methods.