Compound Emotion Recognition of Autistic Children During Meltdown Crisis Based on Deep Spatio-Temporal Analysis of Facial Geometric Features

Salma Kammoun Jarraya,Mohamed Hammami,Marwa Masmoudi

doi:10.1109/access.2020.2986654

Abstract

An important contribution to computer vision applications has been made by recognizing human emotion. Although it is very significant, this work considers the security of autistic people while in meltdown crisis by introducing a new system to warn caregivers through facial expressions detection. A precautionary approach has been taken to deal with meltdown crisis. Certainly, the indications of Meltdown are linked to abnormal facial expressions related to compound emotions. Actually, researchers thought long ago that Human Facial Expressions (HFE) are not able to express more than the seven basics emotions. HFE have been considered by psychologists as very complicated one, which can indicate two or even more emotions known as compound or mixed ones. A few studies have been done concerning Compound Emotion (CE). As well as, many difficult tasks to detect Compound Emotion Recognition (CER). In this paper, we empirically assess a group of deep spatio-temporal geometric features of micro-expressions of autistic children during a meltdown crisis. To achieve this goal, we make a comparison of the CER performance and diverse collections of micro-expressions features to select the features which best differentiates autistic children CE in meltdown crisis from normal state, and the best classifier performance. We record autistic children videos in normal and meltdown crisis using Kinect camera in serious circumstances. The experimental evaluation shows that the deep spatio-temporal geometric features and Recurrent Neural Network RNN with 3 hidden layer using Information Gain Feature Selection methods provide best performance (85.8%).

Highlights

Anoticeable advancement has been achieved since last years in the development of computer vision applications in covering many areas
As to developing the process of identifying the compound emotions of autistic kids in crisis, this is done by making an analysis of their facial expressions, we primarily looked at a detailed and critical literature review of the works that examine the emotional recognition from the facial expression
We discuss works which have to do with basic emotions of people with autism and we present a description of variety of methods which deal with the compound emotions.We are going to study the works that expose the basic emotions of autistic individuals and we are going to present an overview which proposes the different techniques of how to deal with the compound emotions

Summary

Introduction

Anoticeable advancement has been achieved since last years in the development of computer vision applications in covering many areas. The applications’ spectrum is too wide, there are few solutions to assist autistic children. An abnormal social interaction and communication along with limited and recurrent behaviors are what define an autism of ‘‘Autism Spectrum Disorder (ASD). Autistic people have become part of our society. Studies in 2018 [1] point that the rate of autistic people would vary between 90 and 120 persons among 10000, or 1% of the population. Autism can be detected in early age mainly at the age 2 or 3, when children isolate themselves and behave differently. Autistic children can overcome their problem through many facilities. Whereas, the latter are sufficient to satisfy their exceptional needs in autism crisis Autistic children can overcome their problem through many facilities. whereas, the latter are sufficient to satisfy their exceptional needs in autism crisis

Objectives

Results

Discussion

Conclusion