Towards an intelligent assistive system based on augmented reality and serious games

Abstract

Age-related cognitive impairment is generally characterized by gradual memory loss and decision-making difficulties. The aim of this study is to investigate multi-level support and suggest relevant helping means for the elderly with mild cognitive impairment as well as their caregivers as the primary end-users. This work reports preliminary results on an intelligent assistive system, achieved through the integration of Internet of Things, augmented reality, and adaptive fuzzy decision-making methods. The proposed system operates in different modes, including automated and semi-automated modes. The former helps the user complete their daily life activities by showing augmented reality messages or making automatic changes; while the latter allows manual changes after the real-time assessment of the user’s cognitive state based on the augmented reality serious game score. We have also evaluated the accuracy of the serious game score with 37 elderly participants and compared it with users’ paper-based cognitive test results. We further noted that there is an acceptable correlation between the paper-based test and users’ serious game scores. Moreover, we observed that the system response in the semi-automated mode causes less data loss compared with the automated mode, as the number of active devices decreases.