A feasibility study to explore the use of real‐time caregiver reports for people living with dementia at home to label multi‐modal sensor data

Abstract

AbstractBackgroundMulti‐modal sensor systems, consisting of sensors throughout the home, are able to collect data on the person living with dementia’s (PlwD’s) activities. However, caregivers may have challenges interpreting the data, leading to a lack of actionable information. It’s important to get real‐time reports from caregivers to label the sensor data to build relevant machine learning algorithms; yet, this has not been done with PlwD. The aim of this feasibility study was to explore if the use of a multi‐modal sensor system in conjunction with real‐time caregiver reports can provide valuable insights into the daily lives of PlwD.MethodWe developed the MAISON (Multimodal AI‐based Sensor platform for Older Individuals) system and deployed it into the homes of dyads (PlwD and their family caregiver) for 8‐weeks. MAISON includes: a smartphone, a smartwatch to continuously collect heart rate, step, and accelerometer data to infer walking, and running, as well as a smart mattress that can collect sleep time and quality. A digital diary was added to the MAISON system that automatically sends the caregiver 2‐3 pre‐set questions every day about their loved one. Caregivers could also report any significant occurrences throughout the day, as they see fit, through the app.ResultTwo dyads were recruited to date. Only about 15% of data was missing from the sensor system by the end of the study. Caregivers were providing their reports with high adherence. The caregivers’ reports were used to label the sensor data successfully, and upon analysis, we found that caregivers’ reports were reflected in the sensor data and the labels derived from their reports were accurate. Results indicate this study methodology is an innovative approach to utilize the family caregiver’s reports.ConclusionThis is the first longitudinal study to demonstrate the feasibility of a multi‐modal sensor system used by caregivers and PwlD. Findings show that caregivers' reports can be useful sources of data for labeling sensor data. The conjunction of caregiver reports and sensor data will provide valuable information to build accurate machine learning models about behaviors of PlwD which is a foundational step to support dyads at home using technology.