Interactive Question-Posing System for Robot-Assisted Reminiscence From Personal Photographs

Abstract

Reminiscence is a lifelong activity that happens throughout our lifespan. While memories can serve as topics in people’s everyday conversations, recalling the past can also help us build self-esteem and increase our level of happiness. In this paper, we aim to develop a robot companion that helps people to recollect their memories from personal photographs. We focus on how a robot can associate concepts relevant to the content in the photographs and evoke people’s memories by asking questions that are both relatable and engaging. To understand the content of a picture, we applied deep learning techniques in order to recognize events, objects, and scenes in it. Then, these observations and any user utterances are considered in a Markov random field (MRF)-based algorithm that contains common sense knowledge of a number of events, with loopy belief propagation being used to infer possible associated concepts and topics. Afterward, the robot poses appropriate questions about the selected topics, guiding the user to reminisce through conversation. Our results show that the proposed system can pose related and appropriate questions to interact with the user, and has the potential guide the user to recall the past in an organized way.