FlexiSee: flexible configuration, customization, and control of mediated and augmented vision for users of smart eyewear devices

Abstract

Smart eyewear and Augmented Reality technology have been examined closely in the scientific community to provide vision rehabilitation to people with visual impairments as well as augmented vision to people with and without visual impairments for various application scenarios and contexts of use. However, current systems lack flexibility in the configuration and customization of the features and functionalities they present to their users, as we show in this paper by means of a thorough literature review and categorization of prior work on augmented and mediated vision for smart eyewear devices. To address the flexibility aspect that has been missing in prior work, we introduce FlexiSee, an application for smart eyewear devices, such as see-through Augmented Reality glasses and Head-Mounted Mixed Reality Displays, specifically designed to enable flexible configuration, customization, and control of both augmented and mediated vision. FlexiSee achieves this desiderata by implementing visual filters (e.g., color correction, edge highlighting, contrast adjustment, and others) that are coupled with a web-based interface, readily accessible from smartphones, tablets, smartwatches, and other devices with web browsers, where authorized users can specify and apply custom parameters for the visual filters implemented by FlexiSee. We also introduce FlexiSee-DS, a three-dimensional design space for FlexiSee-like applications, that includes mediation & augmentation, user categories, and control design dimensions to specify a variety of FlexiSee-like systems. We show how the dimensions of FlexiSee-DS were applied to inform the design of our FlexiSee system, and we highlight and focus on the distinction between primary users and vision monitors and assistants, where the latter two categories represent new types of users for augmented and mediated vision that have various degrees of control, from their remote locations, over the visual reality delivered to and perceived by the primary users of smart eyewear devices. We conduct a user study to understand the perception of vision monitors and assistants regarding our new FlexiSee concept and system, and we report empirical results about usability aspects (e.g., we found an average SUS score of 75.3 and high ratings for the perceived usefulness of FlexiSee) as well as user feedback and suggestions to inform further developments of FlexiSee-like systems and applications.