Mixed reality or LEGO game play? Fostering social interaction in children with Autism

Abstract

This study extends the previous research in which it has been shown that a mixed reality (MR) system fosters social interaction behaviours (SIBs) in children with Autism Spectrum Condition (ASC). When comparing this system to a LEGO-based non-digital intervention, it has been observed that an MR system effectively mediates a face-to-face play session between a child with ASC and a child without ASC providing new specific advantageous properties (e.g. not being a passive tool, not needing to be guided by the therapist). Considering the newly collected multimodal data totaling to 72 children (36 trials of dyads, child with ASC/child without ASC), a first goal of the present study is to apply detailed statistical inference and machine learning techniques to extensively evaluate the overall effect of this MR system, when compared to the LEGO condition. This goal also includes the analysis of psychophysiological data and allows the context-driven triangulation of the multimodal data which is operationalized by (i) video-coding of SIBs, (ii) psychophysiological data, and (iii) system logs of user-system events. A second goal is to show how SIBs, taking place in these experiences, are influenced by the internal states of the users and the system. SIBs were measured by video-coding overt behaviours (Initiation, Response and Externalization) and with self-reports. Internal states were measured using a wearable device designed by the FuBIntLab (Full-Body Interaction Lab) to acquire: Electrocardiogram (ECG) and Electrodermal Activity (EDA) data. Affective sliders and State Trait Anxiety Scale questionnaires were used as self-reports. Repeated-measures design was chosen with two conditions, the MR environment and the traditional therapy LEGO. The results show that the MR system has a positive effect on SIBs when compared to the LEGO condition, with an added advantage of being more flexible.