User input enrichment via sensing devices

Abstract

Nowadays a user may have many electronic devices. However, these devices suffer from different resource and usage constraints. On one hand, mobile devices, such as smartphones, have short battery life, limited computing power, and small-sized display. On the other hand, more powerful devices such as desktops or smart TVs are bulky and lack motion-related input data. In this paper, we aim to integrate these two types of devices into one computing platform when both devices are accessible to a user, in which their individual advantages are combined for users’ convenience. To this end, we develop a new user-centric paradigm on computing systems, called Application Execution with Sensing Input (AESIP), which can transparently inject sensing data to a powerful yet stationary device using an auxiliary mobile device. Not requiring any modification on the mobile device’s OS and applications, AESIP supports all mobile-specific input data sources, such as touchscreen, gyroscope, and accelerometer. As one design goal of AESIP is to maximize the user’s Quality of Experience (QoE), we tackle several usability challenges and enable richer functionality. We implement a prototype of AESIP on a Nexus 5, a Raspberry Pi and a desktop machine. Our performance evaluation shows that AESIP induces little latency and negligible bandwidth usage, and it significantly increases the battery life of a mobile device. We further conduct a user study to evaluate the usability of AESIP.