Study of Multi-Modal Diary Service Using Spoken Dialogue Agent for Self-Care in Elderly People

Abstract

With Japan's aging population and low birth rate, the self-care of older adults at home has become increasingly in demand. Although the rapid progress of AI-enabled healthcare is developing, diversity and reusability of in-home daily living data to be recorded and analyzed are still challenging points. In our previous study, we have developed a system utilizing spoken dialogue agents for listening to the minds of older adults. The purpose of this study is to propose a multi-modal diary service that records and analyzes dialogue data with a diversified viewpoint. Our key idea is to retrieve the text logs, audio, and images and execute multi-modal visualization. The proposed method describes the main steps, including heterogeneous data extraction, Web API development, multi-modal diary generation, and heterogeneous data analysis. Also, we have discussed the approach to extending heterogeneous data. Using the proposed method, older adults can recollect the past for their own better. In this way, the correct improvement of in-home care and health management for all individuals is promising.