Augmented Visual Intelligence

Abstract

We propose an Augmented Visual Intelligence (AVI) framework to assist human in vision- and memory-related tasks. The AVI framework exploits wearable cameras and ambient computing facilities to empower a user's vision and memory functions by answering four types of queries central to visual activities. In particular, the Extended Visual Memory (EVM) model plays a central role in AVI. Learning of EVM stores view-based visual fragments (VF), which are abstracted into high-level visual schemas (VS), both in the visual long-term memory. During inference, the visual short-term memory plays a key role in the schematic representations of, and the similarity computation between, a visual input and a VF, exemplified from VS when necessary. In this paper, we describe the AVI framework and the EVM model followed by an implementation scenario on assisted living.