Automatic classification of ASD children using appearance-based features from videos

Abstract

Early diagnosis of Autism Spectrum Disorder (ASD) plays a crucial role in the intervention of ASD. Traditionally, an ASD child needs to be diagnosed by a psychiatrist in the hospital, which is expensive, time-consuming, and influenced by expertise. In this paper, we present an objective, convenient, and effective method for classifying ASD children from raw video sequences by integrating the appearance-based features from facial expressions, head pose, and head trajectory. To better extract facial expression features, we propose a novel attention-based facial expression recognition algorithm to focus on key face areas like eyebrows, mouth, etc. Moreover, we use accumulative histogram to individually extract temporal and spatial information from facial expression, head pose and head trajectory of the video sequence. After fusing these three kinds of features, we feed them to Long Short-Term Memory (LSTM) and achieve a classification accuracy of 96.7% on our self-collected ASD video dataset.