Classifying ASD children with LSTM based on raw videos

Abstract

Autism spectrum disorder (ASD) is a serious neurodevelopmental disorder that impairs a child's ability to communicate and interact with others. Usually, recognizing a child with ASD needs the diagnosis by professional doctors. However, it is not only expensive and time-consuming, but also the results are influenced by subjective factors, such as the experience of a doctor. Recently, some methods which identify ASD based on biomarkers have been developed, but there are rarely works specific to raw video data. This paper is the first attempt to help diagnose the children with ASD in raw video data using a deep learning technique. Firstly, in order to investigate different gaze patterns between ASD children and typically developing (TD) children, we track the eye movement in each video by the tracking-learning-detection method. Secondly, we divide these tracking trajectories into two components: (i) the length; and (ii) the angle. Afterwards, we calculate an accumulative histogram for each component. Finally, we adopt three-layer Long Short-Term Memory (LSTM) network for classification. Experimental results on our extended dataset (Ext-Dataset) containing 272 videos captured from 136 ASD children and 136 TD children show the LSTM network outperforms the traditional machine learning methods, e.g., Support Vector Machine, with the improvement of accuracy by 6.2% (from 86.4% to 92.6%). Especially, for ASD, we obtain the sensitivity (the true positive rate, TPR) of 91.9% and the specificity (the true negative rate, TNR) of 93.4%, which demonstrates the effectiveness of our method.