Emotion recognition using speckle pattern analysis and k-nearest neighbors classification

Abstract

Emotion recognition is a basic communication tool in our daily interaction, and the recognition of emotions without contact and with high sensitivity may be very useful for various purposes. This paper presents a preliminary experimental investigation in which emotions of healthy subjects were recognized while they performed facial gestures related to those different emotions. Their faces were illuminated with a few laser spots and the formed back-scattered speckle patterns were analyzed with a camera having proper optics. By analyzing the temporal variation in the spatial distribution of those speckle patterns we estimated the muscles’ contraction–release motion in specific locations. The used data amount for the estimation procedure was less than 1% of the face frame so as to maintain the subjects’ privacy. Moreover, the presented optic method enables the detection of minor movements that cannot be recognized by the naked eye or conventional visual processing. After applying the machine learning k-nearest neighbors algorithm, we succeeded in reaching 89% accuracy in the recognition of emotions for the combination of two classification steps: subject recognition among the participants, and then emotion recognition among three optional emotions: happiness, sadness and neutral expression.