RiSH: A robot-integrated smart home for elderly care

Abstract

This article presents the development of a robot-integrated smart home (RiSH) which can be used for research in assistive technologies for elderly care. The RiSH integrates a home service robot, a home sensor network, a body sensor network, a mobile device, cloud servers, and remote caregivers. A layered architecture is proposed to guide the design and implementation of the RiSH software. Basic service functions are developed to allow the RiSH to recognize human body activity using an inertial measurement unit (IMU) and the home service robot to perceive the environment through audio signals. Based on these functions, we developed two low-level applications: (1) particle filter-based human localization and tracking using wearable motion sensors and distributed binary sensors; (2) Dynamic Bayesian Network-based human activity recognition using microphones and distributed binary sensors. Both applications extend the robot’s perception beyond its onboard sensors. Utilizing the low-level applications, a high-level application is realized that detects and responds to human falls. We conducted experiments in our RiSH testbed to evaluate auditory perception services, human body activity recognition, human position tracking, sound-based human activity monitoring, and fall detection and rescue. Twelve human subjects were asked to conduct daily activities in the testbed and mimic falls. All data of their movement, body activities, and sound events were collected by the robot. Human trajectories were estimated with a root mean square error of less than 0.2 m. The robot was able to recognize 37 human activities through sound events with an average accuracy of 88% and detect falling sounds with an accuracy of 80% at the frame level. The experiments show the operation of the various components in the RiSH and the capabilities of the home service robot in monitoring and assisting the resident.