Using visual lifelogs to automatically characterize everyday activities

Abstract

Visual lifelogging is the term used to describe recording our everyday lives using wearable cameras, for applications which are personal to us and do not involve sharing our recorded data. Current applications of visual lifelogging are built around remembrance or searching for specific events from the past. The purpose of the work reported here is to extend this to allow us to characterize and measure the occurrence of everyday activities of the wearer and in so doing to gain insights into the wearer’s everyday behavior. The methods we use are to capture everyday activities using a wearable camera called SenseCam, and to use an algorithm we have developed which indexes lifelog images by the occurrence of basic semantic concepts. We then use data reduction techniques to automatically generate a profile of the wearer’s everyday behavior and activities. Our algorithm has been evaluated on a large set of concepts investigated from 13 users in a user experiment, and for a group of 16 popular everyday activities we achieve an average F-score of 0.90. Our conclusions are that the technique we have presented for unobtrusively and ambiently characterizing everyday behavior and activities across individuals is of sufficient accuracy to be usable in a range of applications.