Blurring the boundary between direct & indirect mixed mode input environments

Abstract

The current computer input devices are either direct or indirect based on how interactive input data is interpreted by a computing system. Existing research have shown the benefits and limitations of both input modalities, and found that the limits of one input mode are the advantages of the other mode. In this thesis, we explore a new type of input device, which integrates direct and indirect input into a shared input space: mixed input mode. With a mixed mode input device, the two input modes could well complement each other in the shared input space, such that users can freely choose which mode to use to achieve optimized performance. We explore the hardware and software design space of mixed mode input device. On the hardware side, we created three novel prototype devices: 1) LucidCursor—a mixed mode input handheld device; 2) LensMouse—a mixed mode input computer mouse; and Magic Finger—a mixed mode finger-worn input device. We demonstrate how these devices can be built. We also evaluated the performance of the three proposed prototypes from various perspectives through a set of carefully designed user and system evaluations. On the software side, most of the current software user interfaces are only designed and optimized for a certain input mode, e.g. either mouse or finger input. To facilitate the mixed mode input on the existing software UIs, we proposed and evaluated two target expansion techniques—TouchCuts & TouchZoom. These techniques allow touchscreen laptop users to freely choose an input mode without impacting users' experience of the other input mode (e.g. mouse). An important finding of this thesis is that the mixed mode input devices have many add-on benefits, which can significantly improve the user experience of interacting with computer systems.