Tongue-able interfaces: Prototyping and evaluating camera based tongue gesture input system

Abstract

Tongue-computer interaction techniques create a new pathway between human mind and computer, with particular utility for people with upper limb impairment. The high dexterity and resilience of the tongue make it a good candidate for interacting with computers. This paper introduces a new interaction technique, camera-based tongue computer interface (CBTCI), which employs tongue without any direct physical contact required. Through a two-phase study, the CBTCI was evaluated and its interaction problems were identified and discussed. In the first phase, the performance of the CBTCI prototype was evaluated through two user tests. The participants’ behaviors were observed throughout the first phase and analyzed to scaffold the study of the design problems in gesture-based tongue computer interaction. The Phase 2 study investigated the usability problems of CBTCI which were reflected through the user behavior and participants’ feedback; specifically the exploration of referential techniques to make users aware of their tongue position and adjust their gesture. Pros and cons of the referential strategies are discussed to foster future assistive tongue-computer interface design.