Detection of Behaviors of Risk exhibited by People with Dementia with Privacy Protecting Videos

Abstract

AbstractBehavioural symptoms of dementia present a significant risk within long‐term care homes, resulting in difficulties supporting residents and monitoring their safety due to limited staffing resources. Existing video surveillance infrastructure can be used to monitor and automatically detect clinically important behaviours that can put the residents at risk. An anomaly detection approach can be used to detect behaviours of risk from videos due to their infrequent and diverse nature. However, most existing video anomaly detection approaches focus on appearance‐based features, which can put the privacy of a person at risk and is also susceptible to pixel‐based noise. In this study, we explored different privacy‐protecting approaches and present a conditional average neighbour score method to detect behaviours of risks in videos from a dementia care unit. The conditional average neighbour score method considers that behaviours of risk in people with dementia are sometimes momentary, but often they persist over time. In this study, we used semantic segmentation masks approach to protect participants’ privacy by obscuring them in the video. The privacy‐protecting inputs were used to train a customized spatio‐temporal convolutional autoencoder and identify behaviours of risk as anomalies. The video was collected on a dementia care unit and composed of approximately 21 hours of training data (normal activities) and 9 hours of test data (normal activities and behaviours of risk events). In comparison to RGB input (0.822), we obtained an equivalent area under the receiver operating characteristic curve (AUC ROC) performance of 0.823 for the segmentation mask‐based privacy‐protecting approach. Further, using the conditional average neighbour score method, we improved the AUCROC performance to 0.83 for the segmentation mask‐based privacy‐protecting approach. The results signify that it is possible to use surveillance videos from a dementia care unit to detect behaviours of risk as anomalies, and that privacy protecting methods work as well as methods using the raw videos. This research paves the way to improve the quality of life of residents and reduce injuries in residential care homes, while respecting their privacy.